Jul 07 14:59:03.182968 ip-10-0-141-205 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 14:59:03.182981 ip-10-0-141-205 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 14:59:03.182990 ip-10-0-141-205 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 14:59:03.183317 ip-10-0-141-205 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 14:59:13.233325 ip-10-0-141-205 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 14:59:13.233345 ip-10-0-141-205 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 6223d6c96841442daa12f59a70ab2f59 -- Jul 07 15:01:10.457220 ip-10-0-141-205 systemd[1]: Starting Kubernetes Kubelet... Jul 07 15:01:10.857303 ip-10-0-141-205 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:01:10.857303 ip-10-0-141-205 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 15:01:10.857303 ip-10-0-141-205 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:01:10.857303 ip-10-0-141-205 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 15:01:10.857303 ip-10-0-141-205 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:01:10.857990 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.857322 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 15:01:10.861729 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861714 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:01:10.861729 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861728 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861733 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861736 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861739 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861742 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861745 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861748 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861750 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861753 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861756 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861759 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861762 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861764 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861767 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861769 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861772 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861774 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861777 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861780 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861782 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:01:10.861790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861784 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861787 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861790 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861792 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861795 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861798 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861801 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861804 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861806 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861811 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861816 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861819 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861822 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861825 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861828 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861831 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861833 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861836 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861841 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:01:10.862266 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861844 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861847 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861850 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861852 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861855 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861857 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861860 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861863 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861865 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861868 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861870 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861873 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861875 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861878 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861880 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861884 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861887 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861889 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861892 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861895 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:01:10.862730 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861897 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861900 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861902 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861905 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861909 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861911 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861914 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861916 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861919 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861922 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861925 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861928 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861930 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861933 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861949 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861953 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861960 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861964 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861967 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861970 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:01:10.863256 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861973 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861976 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861979 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861981 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861984 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861987 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861990 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861993 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861996 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.861999 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862002 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862004 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862007 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862010 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862013 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862015 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862018 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862021 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862024 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862694 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:01:10.864013 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862700 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862704 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862706 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862709 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862712 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862714 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862717 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862720 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862722 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862725 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862728 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862730 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862733 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862736 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862739 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862742 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862744 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862747 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862750 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862753 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:01:10.864611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862756 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862759 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862762 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862764 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862767 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862770 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862772 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862775 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862778 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862780 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862784 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862788 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862792 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862795 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862798 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862801 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862805 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862807 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862810 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:01:10.865150 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862813 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862815 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862818 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862821 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862823 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862826 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862829 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862831 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862834 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862837 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862839 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862842 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862846 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862859 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862862 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862867 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862871 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862874 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862877 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:01:10.865633 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862879 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862882 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862885 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862888 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862890 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862894 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862897 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862900 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862902 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862905 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862909 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862911 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862914 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862916 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862918 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862921 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862924 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862927 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862930 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862932 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:01:10.866111 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862947 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862951 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862953 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862956 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862959 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862962 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862965 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862968 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862971 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862974 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862976 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862979 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862982 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862984 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862987 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862989 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862992 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862994 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.862997 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.863000 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:01:10.866606 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864517 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864629 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864635 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864640 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864644 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864647 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864650 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864655 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864658 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864661 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864664 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864667 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864670 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864672 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864677 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864681 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864684 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864687 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864690 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:01:10.867113 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864692 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864695 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864698 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864701 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864704 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864706 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864709 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864712 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864715 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864718 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864721 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864724 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864726 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864730 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864733 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864737 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864739 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864742 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864745 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864747 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:01:10.867560 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864750 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864754 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864757 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864760 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864762 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864766 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864768 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864771 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864774 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864776 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864779 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864782 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864784 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864786 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864789 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864792 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864794 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864797 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864800 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864802 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:01:10.868738 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864805 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864807 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864810 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864813 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864816 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864820 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864824 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864828 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864830 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864833 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864835 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864838 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864841 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864843 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864846 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864848 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864851 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864853 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864856 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:01:10.869232 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864858 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864861 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864866 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864871 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864874 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864876 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864879 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864882 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864885 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864888 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864890 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864893 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864895 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864898 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864900 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864903 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864906 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864909 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864911 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864915 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:01:10.869686 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.864917 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865003 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865011 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865020 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865027 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865031 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865034 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865039 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865043 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865046 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865049 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865053 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865056 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865059 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865063 2565 flags.go:64] FLAG: --cgroup-root="" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865065 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865068 2565 flags.go:64] FLAG: --client-ca-file="" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865071 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865076 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865081 2565 flags.go:64] FLAG: --cluster-domain="" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865084 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865087 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865090 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865094 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 15:01:10.870187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865098 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865100 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865104 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865107 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865110 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865113 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865116 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865122 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865125 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865130 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865133 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865136 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865138 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865141 2565 flags.go:64] FLAG: --enable-server="true" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865144 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865149 2565 flags.go:64] FLAG: --event-burst="100" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865152 2565 flags.go:64] FLAG: --event-qps="50" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865155 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865159 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865162 2565 flags.go:64] FLAG: --eviction-hard="" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865166 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865169 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865172 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865175 2565 flags.go:64] FLAG: --eviction-soft="" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865177 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 15:01:10.870752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865180 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865183 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865187 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865190 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865193 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865196 2565 flags.go:64] FLAG: --feature-gates="" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865200 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865203 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865207 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865209 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865212 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865215 2565 flags.go:64] FLAG: --help="false" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865218 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-141-205.ec2.internal" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865221 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865226 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865229 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865233 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865236 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865239 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865242 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865245 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865248 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865251 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865254 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 15:01:10.871370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865257 2565 flags.go:64] FLAG: --kube-reserved="" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865260 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865262 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865265 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865268 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865271 2565 flags.go:64] FLAG: --lock-file="" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865273 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865276 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865279 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865285 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865288 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865292 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865294 2565 flags.go:64] FLAG: --logging-format="text" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865297 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865300 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865303 2565 flags.go:64] FLAG: --manifest-url="" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865306 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865310 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865314 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865318 2565 flags.go:64] FLAG: --max-pods="110" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865321 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865324 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865329 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865332 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865335 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 15:01:10.871954 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865338 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865340 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865347 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865350 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865353 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865357 2565 flags.go:64] FLAG: --pod-cidr="" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865360 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865389 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865393 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865396 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865400 2565 flags.go:64] FLAG: --port="10250" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865403 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865405 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0929c0973224c6dc9" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865409 2565 flags.go:64] FLAG: --qos-reserved="" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865412 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865415 2565 flags.go:64] FLAG: --register-node="true" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865417 2565 flags.go:64] FLAG: --register-with-taints="" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865421 2565 flags.go:64] FLAG: --registry-burst="10" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865424 2565 flags.go:64] FLAG: --registry-qps="5" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865427 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865430 2565 flags.go:64] FLAG: --reserved-memory="" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865434 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865437 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865440 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865443 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 15:01:10.872564 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865446 2565 flags.go:64] FLAG: --runonce="false" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865449 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865452 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865454 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865459 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865462 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865465 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865468 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865471 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865473 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865476 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865479 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865482 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865485 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865488 2565 flags.go:64] FLAG: --system-cgroups="" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865491 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865498 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865500 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865503 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865508 2565 flags.go:64] FLAG: --tls-min-version="" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865510 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865513 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865516 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865519 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865524 2565 flags.go:64] FLAG: --v="2" Jul 07 15:01:10.873191 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865528 2565 flags.go:64] FLAG: --version="false" Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865532 2565 flags.go:64] FLAG: --vmodule="" Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865536 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.865540 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865666 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865671 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865674 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865677 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865680 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865682 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865685 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865689 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865692 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865695 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865697 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865700 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865702 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865705 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865708 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865710 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865713 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:01:10.873783 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865716 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865719 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865721 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865724 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865726 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865729 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865731 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865734 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865737 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865739 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865744 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865747 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865750 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865752 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865756 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865758 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865761 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865764 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865767 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865769 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:01:10.874307 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865771 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865775 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865779 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865782 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865785 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865788 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865792 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865795 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865798 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865801 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865804 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865806 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865810 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865814 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865817 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865820 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865822 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865825 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865827 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:01:10.874790 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865830 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865832 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865835 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865839 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865842 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865844 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865847 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865849 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865853 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865855 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865858 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865860 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865863 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865865 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865868 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865871 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865873 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865876 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865878 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:01:10.875283 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865881 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865883 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865886 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865889 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865891 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865894 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865897 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865899 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865902 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865904 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865906 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865909 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865912 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865914 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865917 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865919 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865923 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865927 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865930 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865933 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:01:10.875755 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865948 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865952 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865955 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.865958 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.866786 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.873380 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.873396 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.873415 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.873422 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873478 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873483 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873487 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873490 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873493 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873496 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:01:10.876245 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873499 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873502 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873504 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873507 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873510 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873513 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873516 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873520 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873524 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873528 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873531 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873534 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873537 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873540 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873543 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873547 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873550 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873553 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:01:10.876699 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873556 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873559 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873561 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873564 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873567 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873571 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873574 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873577 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873580 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873582 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873585 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873588 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873590 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873593 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873595 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873598 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873601 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873603 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873606 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873609 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:01:10.877145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873612 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873614 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873617 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873619 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873622 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873625 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873627 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873630 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873633 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873635 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873638 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873640 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873643 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873646 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873649 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873652 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873655 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873657 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873661 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873664 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:01:10.877622 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873667 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873669 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873672 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873674 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873677 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873680 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873682 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873685 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873688 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873690 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873693 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873695 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873698 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873701 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873703 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873706 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873709 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873711 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873714 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873716 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:01:10.878142 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873719 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873722 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873725 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873728 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873730 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873733 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873737 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873740 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873743 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873745 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873748 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873751 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873754 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873757 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873759 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:01:10.878630 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.873765 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873893 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873898 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873901 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873904 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873907 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873909 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873912 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873915 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873918 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873920 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873923 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873925 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873928 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873931 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873933 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873951 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873955 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873959 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873963 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:01:10.879112 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873965 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873969 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873971 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873974 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873977 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873980 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873983 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873986 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873988 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873992 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873995 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.873998 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874000 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874003 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874006 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874008 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874011 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874014 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874016 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:01:10.879591 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874018 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874021 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874024 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874026 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874029 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874031 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874034 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874036 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874039 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874042 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874046 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874049 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874052 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874055 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874058 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874061 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874064 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874066 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874070 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:01:10.880077 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874073 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874076 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874080 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874083 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874086 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874089 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874092 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874094 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874096 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874099 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874102 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874104 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874106 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874109 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874112 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874114 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874117 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874119 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874122 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874124 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:01:10.880544 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874127 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874129 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874132 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874134 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874137 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874139 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874142 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874145 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874147 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874150 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874152 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874156 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874159 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874162 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874165 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874167 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874170 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874172 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874175 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874177 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:01:10.881021 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874180 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:01:10.881482 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:10.874182 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:01:10.881482 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.874187 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:01:10.881482 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.874298 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 15:01:10.881482 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.876246 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 15:01:10.881482 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.877614 2565 server.go:1019] "Starting client certificate rotation" Jul 07 15:01:10.881482 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.877712 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 15:01:10.881482 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.878333 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 15:01:10.881482 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.880086 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 15:01:10.883865 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.883848 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 15:01:10.898234 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.898213 2565 log.go:25] "Validated CRI v1 runtime API" Jul 07 15:01:10.900969 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.900954 2565 log.go:25] "Validated CRI v1 image API" Jul 07 15:01:10.901814 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.901783 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 15:01:10.907042 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.907023 2565 fs.go:135] Filesystem UUIDs: map[4018a690-39be-4fdc-8a76-48d5e5eb91aa:/dev/nvme0n1p3 41cef6a4-0ec7-4ae7-a627-cab86b2cbfc8:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 07 15:01:10.907105 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.907041 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 15:01:10.913617 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.913511 2565 manager.go:216] Machine: {Timestamp:2026-07-07 15:01:10.912055232 +0000 UTC m=+0.337443612 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098361 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2587862010ed3192e74922c0c7b514 SystemUUID:ec258786-2010-ed31-92e7-4922c0c7b514 BootID:6223d6c9-6841-442d-aa12-f59a70ab2f59 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cf:ab:ef:51:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cf:ab:ef:51:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:cb:37:ed:a2:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 15:01:10.913617 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.913614 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 15:01:10.913726 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.913696 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 15:01:10.918797 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.918768 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 15:01:10.918965 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.918797 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-205.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 15:01:10.920153 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.920141 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 15:01:10.920204 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.920157 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 15:01:10.920204 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.920166 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 15:01:10.920772 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.920763 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 15:01:10.921765 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.921749 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 15:01:10.922080 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.922069 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 15:01:10.925369 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.925358 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 07 15:01:10.925535 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.925526 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 15:01:10.925574 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.925555 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 07 15:01:10.925574 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.925565 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 15:01:10.925638 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.925576 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 15:01:10.925638 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.925586 2565 kubelet.go:398] "Adding apiserver pod source" Jul 07 15:01:10.925638 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.925594 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 15:01:10.925968 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.925956 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 15:01:10.926013 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.925975 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 15:01:10.928409 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.928386 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 15:01:10.930593 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.930580 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 15:01:10.930632 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.930610 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 15:01:10.931727 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.931716 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 15:01:10.931760 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.931731 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 15:01:10.931760 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.931737 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 15:01:10.931760 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.931743 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 15:01:10.931760 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.931751 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 15:01:10.931760 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.931756 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 15:01:10.931892 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.931765 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 15:01:10.931892 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.931771 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 15:01:10.931892 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.931778 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 15:01:10.931892 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.931783 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 15:01:10.931892 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.931791 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 15:01:10.931892 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.931796 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 15:01:10.932696 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.932687 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 15:01:10.932726 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.932697 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 15:01:10.936033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.936019 2565 server.go:1268] "Started kubelet" Jul 07 15:01:10.936159 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.936107 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 15:01:10.936231 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.936129 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 15:01:10.936231 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.936217 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 15:01:10.936959 ip-10-0-141-205 systemd[1]: Started Kubernetes Kubelet. Jul 07 15:01:10.937910 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.937885 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 15:01:10.938538 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:10.938517 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 15:01:10.938615 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.938524 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-205.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 15:01:10.938615 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:10.938548 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-205.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 15:01:10.938686 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.938624 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 07 15:01:10.940903 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:10.940887 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 15:01:10.944580 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.944564 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 15:01:10.944649 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.944588 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 15:01:10.944774 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:10.944755 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:10.945214 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.945194 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 15:01:10.945304 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.945231 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 15:01:10.945304 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.945242 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 15:01:10.945388 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.945337 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 07 15:01:10.945388 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.945348 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 07 15:01:10.945906 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:10.945054 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-205.ec2.internal.18c00a05719da1cc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-205.ec2.internal,UID:ip-10-0-141-205.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-205.ec2.internal,},FirstTimestamp:2026-07-07 15:01:10.935986636 +0000 UTC m=+0.361375019,LastTimestamp:2026-07-07 15:01:10.935986636 +0000 UTC m=+0.361375019,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-205.ec2.internal,}" Jul 07 15:01:10.946010 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.945969 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 15:01:10.946010 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.945984 2565 factory.go:55] Registering systemd factory Jul 07 15:01:10.946010 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.945994 2565 factory.go:223] Registration of the systemd container factory successfully Jul 07 15:01:10.946217 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.946204 2565 factory.go:153] Registering CRI-O factory Jul 07 15:01:10.946252 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.946222 2565 factory.go:223] Registration of the crio container factory successfully Jul 07 15:01:10.946283 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.946268 2565 factory.go:103] Registering Raw factory Jul 07 15:01:10.946315 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.946283 2565 manager.go:1196] Started watching for new ooms in manager Jul 07 15:01:10.946951 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.946916 2565 manager.go:349] Starting recovery of all containers Jul 07 15:01:10.951963 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.951925 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4l995" Jul 07 15:01:10.954643 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:10.954614 2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-205.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 15:01:10.954643 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:10.954631 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 15:01:10.959896 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.959878 2565 manager.go:354] Recovery completed Jul 07 15:01:10.961150 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:10.961103 2565 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 07 15:01:10.962229 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.962212 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4l995" Jul 07 15:01:10.964220 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.964203 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:01:10.966045 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.966032 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:01:10.966106 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.966058 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:01:10.966106 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.966068 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:01:10.966592 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.966580 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 15:01:10.966645 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.966591 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:01:10.966645 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.966593 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 15:01:10.966715 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.966656 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 07 15:01:10.967705 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.967692 2565 policy_none.go:49] "None policy: Start" Jul 07 15:01:10.967741 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.967709 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 15:01:10.967741 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.967720 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 15:01:10.968122 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.968107 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:01:10.968189 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:10.968061 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-205.ec2.internal.18c00a05736848aa default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-205.ec2.internal,UID:ip-10-0-141-205.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-205.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-205.ec2.internal,},FirstTimestamp:2026-07-07 15:01:10.966044842 +0000 UTC m=+0.391433222,LastTimestamp:2026-07-07 15:01:10.966044842 +0000 UTC m=+0.391433222,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-205.ec2.internal,}" Jul 07 15:01:10.968189 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.968132 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:01:10.968189 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.968142 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:01:10.968723 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.968711 2565 policy_none.go:47] "Start" Jul 07 15:01:10.999486 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.999468 2565 manager.go:337] "Starting Device Plugin manager" Jul 07 15:01:10.999569 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:10.999505 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 15:01:10.999569 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.999518 2565 server.go:87] "Starting device plugin registration server" Jul 07 15:01:10.999767 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.999754 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 15:01:10.999821 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.999769 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 15:01:10.999910 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.999893 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 15:01:10.999986 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.999975 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 15:01:10.999986 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:10.999986 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 15:01:11.000486 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.000467 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 15:01:11.000552 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.000520 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:11.096417 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.096367 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 15:01:11.097530 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.097512 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 15:01:11.097616 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.097549 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 15:01:11.097616 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.097572 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 15:01:11.097716 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.097609 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 15:01:11.099989 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.099975 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:01:11.100627 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.100606 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:01:11.100808 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.100796 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:01:11.100852 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.100824 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:01:11.100852 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.100834 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:01:11.100920 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.100862 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.109044 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.108997 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.109044 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.109020 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-205.ec2.internal\": node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:11.132126 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.132099 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:11.198617 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.198588 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-205.ec2.internal"] Jul 07 15:01:11.198744 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.198669 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:01:11.199654 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.199635 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:01:11.199757 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.199671 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:01:11.199757 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.199685 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:01:11.201056 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.201040 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:01:11.201179 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.201164 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.201225 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.201193 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:01:11.201783 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.201767 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:01:11.201868 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.201783 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:01:11.201868 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.201797 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:01:11.201868 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.201804 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:01:11.201868 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.201811 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:01:11.201868 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.201819 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:01:11.203002 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.202987 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.203079 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.203016 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:01:11.203886 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.203870 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:01:11.203960 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.203899 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:01:11.203960 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.203912 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:01:11.232252 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.232227 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:11.233600 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.233586 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-205.ec2.internal\" not found" node="ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.237900 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.237886 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-205.ec2.internal\" not found" node="ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.247111 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.247093 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6cc11541a599eef8412e2d8adc913c1e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal\" (UID: \"6cc11541a599eef8412e2d8adc913c1e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.247172 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.247119 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cc11541a599eef8412e2d8adc913c1e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal\" (UID: \"6cc11541a599eef8412e2d8adc913c1e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.247172 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.247138 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0a252f3cefa1fdcdb223e8cd15c86c48-config\") pod \"kube-apiserver-proxy-ip-10-0-141-205.ec2.internal\" (UID: \"0a252f3cefa1fdcdb223e8cd15c86c48\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.332879 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.332845 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:11.348300 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.348271 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6cc11541a599eef8412e2d8adc913c1e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal\" (UID: \"6cc11541a599eef8412e2d8adc913c1e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.348377 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.348306 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cc11541a599eef8412e2d8adc913c1e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal\" (UID: \"6cc11541a599eef8412e2d8adc913c1e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.348377 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.348331 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0a252f3cefa1fdcdb223e8cd15c86c48-config\") pod \"kube-apiserver-proxy-ip-10-0-141-205.ec2.internal\" (UID: \"0a252f3cefa1fdcdb223e8cd15c86c48\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.348377 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.348342 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6cc11541a599eef8412e2d8adc913c1e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal\" (UID: \"6cc11541a599eef8412e2d8adc913c1e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.348481 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.348377 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0a252f3cefa1fdcdb223e8cd15c86c48-config\") pod \"kube-apiserver-proxy-ip-10-0-141-205.ec2.internal\" (UID: \"0a252f3cefa1fdcdb223e8cd15c86c48\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.348481 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.348411 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cc11541a599eef8412e2d8adc913c1e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal\" (UID: \"6cc11541a599eef8412e2d8adc913c1e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.433718 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.433660 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:11.534201 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.534159 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:11.537313 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.537296 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.541527 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.541497 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-205.ec2.internal" Jul 07 15:01:11.634712 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.634677 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:11.735226 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.735158 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:11.835657 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.835628 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:11.838412 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.838396 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:01:11.876972 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.876924 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 15:01:11.877428 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.877082 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 15:01:11.877428 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.877130 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 15:01:11.900713 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.899480 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:01:11.936003 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:11.935972 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:11.964448 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.964419 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 14:56:10 +0000 UTC" deadline="2028-01-08 22:35:45.443180655 +0000 UTC" Jul 07 15:01:11.964448 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:11.964446 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13207h34m33.478737917s" Jul 07 15:01:12.036819 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:12.036743 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:12.137326 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:12.137295 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:12.237835 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:12.237809 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:12.336136 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:12.336107 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a252f3cefa1fdcdb223e8cd15c86c48.slice/crio-3fd7451bf1c689ae8d1c5af103d1386c22ca54874a10f5096f0a7ff3990441ff WatchSource:0}: Error finding container 3fd7451bf1c689ae8d1c5af103d1386c22ca54874a10f5096f0a7ff3990441ff: Status 404 returned error can't find the container with id 3fd7451bf1c689ae8d1c5af103d1386c22ca54874a10f5096f0a7ff3990441ff Jul 07 15:01:12.336568 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:12.336547 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc11541a599eef8412e2d8adc913c1e.slice/crio-d671cd1bb3219c9ce8234681888ddb12a76f096f5f09663c7b97b25d5bbbcb67 WatchSource:0}: Error finding container d671cd1bb3219c9ce8234681888ddb12a76f096f5f09663c7b97b25d5bbbcb67: Status 404 returned error can't find the container with id d671cd1bb3219c9ce8234681888ddb12a76f096f5f09663c7b97b25d5bbbcb67 Jul 07 15:01:12.338698 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:12.338682 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:12.341415 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.341401 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:01:12.438797 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:12.438756 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-205.ec2.internal\" not found" Jul 07 15:01:12.452887 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.452860 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:01:12.545003 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.544981 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-205.ec2.internal" Jul 07 15:01:12.553117 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.553100 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 15:01:12.554309 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.554297 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal" Jul 07 15:01:12.569179 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.569162 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 15:01:12.743252 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.743177 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:01:12.926038 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.926003 2565 apiserver.go:52] "Watching apiserver" Jul 07 15:01:12.933691 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.933666 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 15:01:12.934112 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.934071 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-66m5k","kube-system/konnectivity-agent-chsnd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal","openshift-multus/multus-kbdqw","openshift-multus/network-metrics-daemon-kfp7w","openshift-network-diagnostics/network-check-target-j52v9","openshift-network-operator/iptables-alerter-pqc4p","kube-system/kube-apiserver-proxy-ip-10-0-141-205.ec2.internal","openshift-cluster-node-tuning-operator/tuned-9z6z7","openshift-dns/node-resolver-6wgcw","openshift-image-registry/node-ca-ssg6q","openshift-multus/multus-additional-cni-plugins-qz6xz","openshift-ovn-kubernetes/ovnkube-node-snw9m"] Jul 07 15:01:12.935801 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.935779 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.937035 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.937007 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:12.938355 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.938339 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:12.938977 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.938959 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 15:01:12.939364 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.939225 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:01:12.939364 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.939266 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xcr2x\"" Jul 07 15:01:12.939502 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.939433 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 15:01:12.939650 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.939633 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 15:01:12.941917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.940001 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 15:01:12.941917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.940025 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tfmx7\"" Jul 07 15:01:12.941917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.940245 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 15:01:12.941917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.940381 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.941917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.940429 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 15:01:12.941917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.941159 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 15:01:12.941917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.941161 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 15:01:12.941917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.941388 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 15:01:12.941917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.941576 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hzdc8\"" Jul 07 15:01:12.943121 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.943105 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x8xkk\"" Jul 07 15:01:12.943250 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.943210 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 15:01:12.943316 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.943301 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 15:01:12.943445 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.943396 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 15:01:12.943565 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.943549 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 15:01:12.943667 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.943638 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 15:01:12.944471 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.944448 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 15:01:12.944592 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.944567 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.945139 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.945122 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 15:01:12.946341 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.946323 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:12.946440 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:12.946411 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:12.947667 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.947646 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 15:01:12.947868 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.947748 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6hj2w\"" Jul 07 15:01:12.948884 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.948815 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pqc4p" Jul 07 15:01:12.950182 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.950162 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:12.950357 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:12.950332 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:12.951608 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.951440 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zdfqh\"" Jul 07 15:01:12.951608 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.951512 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 15:01:12.951608 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.951440 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 15:01:12.951813 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.951713 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:01:12.952598 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.952239 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-chsnd" Jul 07 15:01:12.953916 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.953900 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:12.954021 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:12.953984 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:12.954732 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.954713 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zqgpb\"" Jul 07 15:01:12.954864 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.954839 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 15:01:12.954959 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.954875 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 15:01:12.955918 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.955727 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6wgcw" Jul 07 15:01:12.955918 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.955868 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-node-log\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.955918 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.955897 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-os-release\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.956095 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.955956 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-var-lib-cni-multus\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.956095 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.955984 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-var-lib-kubelet\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.956095 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956008 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl9hx\" (UniqueName: \"kubernetes.io/projected/1443215c-f357-4e92-a03d-ee65a659522d-kube-api-access-cl9hx\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.956095 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956030 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:12.956095 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956052 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-etc-openvswitch\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.956095 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956087 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-cni-bin\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.956344 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956111 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-run-multus-certs\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.956344 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956134 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8n2n\" (UniqueName: \"kubernetes.io/projected/ecd9d42a-2391-4e4e-b2e1-1f3252628680-kube-api-access-f8n2n\") pod \"iptables-alerter-pqc4p\" (UID: \"ecd9d42a-2391-4e4e-b2e1-1f3252628680\") " pod="openshift-network-operator/iptables-alerter-pqc4p" Jul 07 15:01:12.956344 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956159 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x2m7\" (UniqueName: \"kubernetes.io/projected/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-kube-api-access-2x2m7\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:12.956344 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956184 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-tuned\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.956344 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956207 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrzgk\" (UniqueName: \"kubernetes.io/projected/3dece03e-eb3a-4661-855f-f37fca6d1923-kube-api-access-zrzgk\") pod \"network-metrics-daemon-kfp7w\" (UID: \"3dece03e-eb3a-4661-855f-f37fca6d1923\") " pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:12.956344 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956231 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-systemd-units\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.956344 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956250 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-run-netns\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.956344 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956268 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ecd9d42a-2391-4e4e-b2e1-1f3252628680-host-slash\") pod \"iptables-alerter-pqc4p\" (UID: \"ecd9d42a-2391-4e4e-b2e1-1f3252628680\") " pod="openshift-network-operator/iptables-alerter-pqc4p" Jul 07 15:01:12.956344 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956291 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-cnibin\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:12.956344 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956314 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-kubelet\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.956344 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956339 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-run-k8s-cni-cncf-io\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956361 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-run-ovn\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956383 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-system-cni-dir\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956406 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1443215c-f357-4e92-a03d-ee65a659522d-multus-daemon-config\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956430 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ecd9d42a-2391-4e4e-b2e1-1f3252628680-iptables-alerter-script\") pod \"iptables-alerter-pqc4p\" (UID: \"ecd9d42a-2391-4e4e-b2e1-1f3252628680\") " pod="openshift-network-operator/iptables-alerter-pqc4p" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956456 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-sysctl-d\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956481 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rdh5\" (UniqueName: \"kubernetes.io/projected/eaeccbf3-20f1-42cd-b6df-814ce481cb63-kube-api-access-4rdh5\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956505 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-ovnkube-script-lib\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956546 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-lib-modules\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956569 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-ovn-node-metrics-cert\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956602 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-run-netns\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956625 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-socket-dir\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956680 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-sysconfig\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956731 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-var-lib-kubelet\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.956822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956760 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prbmp\" (UniqueName: \"kubernetes.io/projected/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-kube-api-access-prbmp\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956872 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-os-release\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956914 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-etc-selinux\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.956963 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs\") pod \"network-metrics-daemon-kfp7w\" (UID: \"3dece03e-eb3a-4661-855f-f37fca6d1923\") " pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957012 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-kubernetes\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957066 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-log-socket\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957095 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-cni-netd\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957124 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-multus-conf-dir\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957152 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-etc-kubernetes\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957192 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-registration-dir\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957234 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-system-cni-dir\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957262 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-cni-binary-copy\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957304 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957339 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-env-overrides\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957371 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:12.957422 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957410 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-modprobe-d\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957452 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-sysctl-conf\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957480 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-sys\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957520 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-var-lib-cni-bin\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957542 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-run\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957565 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-host\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957586 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-slash\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957611 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-run-openvswitch\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957635 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2r72\" (UniqueName: \"kubernetes.io/projected/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-kube-api-access-h2r72\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957681 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-multus-socket-dir-parent\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957704 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-hostroot\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957725 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-sys-fs\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957748 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-var-lib-openvswitch\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957774 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-run-ovn-kubernetes\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957797 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957820 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-device-dir\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:12.958089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957847 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-run-systemd\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.958832 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957872 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.958832 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957896 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-cnibin\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.958832 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957917 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-systemd\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.958832 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957953 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 15:01:12.958832 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.957954 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-tmp\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:12.958832 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.958122 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-multus-cni-dir\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.958832 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.958176 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1443215c-f357-4e92-a03d-ee65a659522d-cni-binary-copy\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:12.958832 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.958189 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4ptwx\"" Jul 07 15:01:12.958832 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.958201 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-ovnkube-config\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:12.958832 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.958304 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 15:01:12.958832 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.958612 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ssg6q" Jul 07 15:01:12.961381 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.961114 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 15:01:12.961381 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.961131 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 15:01:12.961381 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.961192 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 15:01:12.961381 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.961198 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 15:01:12.961621 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.961423 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vg527\"" Jul 07 15:01:12.981328 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.981305 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6mx96" Jul 07 15:01:12.989784 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:12.989765 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6mx96" Jul 07 15:01:13.050411 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.047593 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058420 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-sysconfig\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058458 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-var-lib-kubelet\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058486 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prbmp\" (UniqueName: \"kubernetes.io/projected/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-kube-api-access-prbmp\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058560 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-sysconfig\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058564 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-var-lib-kubelet\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058595 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-os-release\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058645 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-etc-selinux\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058654 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-os-release\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058673 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs\") pod \"network-metrics-daemon-kfp7w\" (UID: \"3dece03e-eb3a-4661-855f-f37fca6d1923\") " pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058699 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-kubernetes\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058723 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-log-socket\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058758 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-cni-netd\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058763 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-etc-selinux\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058791 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-multus-conf-dir\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058792 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-kubernetes\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058807 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-cni-netd\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.058836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058818 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-etc-kubernetes\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058832 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-log-socket\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058845 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/395c28b2-8a48-403f-81b0-51e8e602646d-host\") pod \"node-ca-ssg6q\" (UID: \"395c28b2-8a48-403f-81b0-51e8e602646d\") " pod="openshift-image-registry/node-ca-ssg6q" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058860 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-multus-conf-dir\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058878 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-registration-dir\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058884 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-etc-kubernetes\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.058798 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058902 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-system-cni-dir\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058919 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-registration-dir\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-cni-binary-copy\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.058971 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs podName:3dece03e-eb3a-4661-855f-f37fca6d1923 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:13.558934342 +0000 UTC m=+2.984322714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs") pod "network-metrics-daemon-kfp7w" (UID: "3dece03e-eb3a-4661-855f-f37fca6d1923") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.058974 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-system-cni-dir\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059001 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059025 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-env-overrides\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059050 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb9ql\" (UniqueName: \"kubernetes.io/projected/bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d-kube-api-access-rb9ql\") pod \"node-resolver-6wgcw\" (UID: \"bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d\") " pod="openshift-dns/node-resolver-6wgcw" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059078 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059102 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-modprobe-d\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.059756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059126 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-sysctl-conf\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059148 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-sys\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059171 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-var-lib-cni-bin\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059191 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-run\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059213 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-host\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059236 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-slash\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059259 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-run-openvswitch\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059284 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2r72\" (UniqueName: \"kubernetes.io/projected/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-kube-api-access-h2r72\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059280 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059309 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-multus-socket-dir-parent\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059358 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-run\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059369 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-multus-socket-dir-parent\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059389 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-hostroot\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059415 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-sys-fs\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059441 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-var-lib-openvswitch\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059448 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-slash\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059465 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-run-ovn-kubernetes\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059485 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-run-openvswitch\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.060626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059492 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d-tmp-dir\") pod \"node-resolver-6wgcw\" (UID: \"bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d\") " pod="openshift-dns/node-resolver-6wgcw" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059517 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vgkv\" (UniqueName: \"kubernetes.io/projected/395c28b2-8a48-403f-81b0-51e8e602646d-kube-api-access-5vgkv\") pod \"node-ca-ssg6q\" (UID: \"395c28b2-8a48-403f-81b0-51e8e602646d\") " pod="openshift-image-registry/node-ca-ssg6q" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059546 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-env-overrides\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059552 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-cni-binary-copy\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059418 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-host\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059548 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059615 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-device-dir\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059622 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-var-lib-openvswitch\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059630 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-sys-fs\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059638 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-run-systemd\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059660 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-run-ovn-kubernetes\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059664 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059693 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-cnibin\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059720 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-device-dir\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059735 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gkgs\" (UniqueName: \"kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs\") pod \"network-check-target-j52v9\" (UID: \"8b6865e3-214f-4260-87bd-66c295f1e763\") " pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059758 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-sys\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059761 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-systemd\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.061513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059793 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-modprobe-d\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059798 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059813 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-systemd\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059815 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-tmp\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059852 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-multus-cni-dir\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059877 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1443215c-f357-4e92-a03d-ee65a659522d-cni-binary-copy\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059901 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae612123-dc47-43fd-b0aa-93d175975cd6-dbus\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059925 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-ovnkube-config\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059927 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-sysctl-conf\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059966 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059992 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-node-log\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.059996 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-var-lib-cni-bin\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060014 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-os-release\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060025 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060032 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-run-systemd\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060060 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-hostroot\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060065 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060054 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-var-lib-cni-multus\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.062280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060095 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060116 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-node-log\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060141 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-os-release\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060126 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-multus-cni-dir\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060229 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-var-lib-cni-multus\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060260 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-var-lib-kubelet\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060271 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-cnibin\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060291 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl9hx\" (UniqueName: \"kubernetes.io/projected/1443215c-f357-4e92-a03d-ee65a659522d-kube-api-access-cl9hx\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060330 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-var-lib-kubelet\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060354 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060380 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-etc-openvswitch\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060409 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-cni-bin\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060439 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-run-multus-certs\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060462 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8n2n\" (UniqueName: \"kubernetes.io/projected/ecd9d42a-2391-4e4e-b2e1-1f3252628680-kube-api-access-f8n2n\") pod \"iptables-alerter-pqc4p\" (UID: \"ecd9d42a-2391-4e4e-b2e1-1f3252628680\") " pod="openshift-network-operator/iptables-alerter-pqc4p" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060489 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae612123-dc47-43fd-b0aa-93d175975cd6-kubelet-config\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060513 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x2m7\" (UniqueName: \"kubernetes.io/projected/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-kube-api-access-2x2m7\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060540 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-tuned\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.063060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060573 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrzgk\" (UniqueName: \"kubernetes.io/projected/3dece03e-eb3a-4661-855f-f37fca6d1923-kube-api-access-zrzgk\") pod \"network-metrics-daemon-kfp7w\" (UID: \"3dece03e-eb3a-4661-855f-f37fca6d1923\") " pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060581 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-ovnkube-config\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060597 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-systemd-units\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060642 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-cni-bin\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060645 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060678 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1443215c-f357-4e92-a03d-ee65a659522d-cni-binary-copy\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060688 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-run-netns\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060662 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-run-multus-certs\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ecd9d42a-2391-4e4e-b2e1-1f3252628680-host-slash\") pod \"iptables-alerter-pqc4p\" (UID: \"ecd9d42a-2391-4e4e-b2e1-1f3252628680\") " pod="openshift-network-operator/iptables-alerter-pqc4p" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060754 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-systemd-units\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060757 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d-hosts-file\") pod \"node-resolver-6wgcw\" (UID: \"bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d\") " pod="openshift-dns/node-resolver-6wgcw" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060789 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-cnibin\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060792 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-etc-openvswitch\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060840 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-cnibin\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060866 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-kubelet\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060901 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-kubelet\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060955 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-host-run-netns\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.060989 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ecd9d42a-2391-4e4e-b2e1-1f3252628680-host-slash\") pod \"iptables-alerter-pqc4p\" (UID: \"ecd9d42a-2391-4e4e-b2e1-1f3252628680\") " pod="openshift-network-operator/iptables-alerter-pqc4p" Jul 07 15:01:13.063828 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061020 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-run-k8s-cni-cncf-io\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061058 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-run-ovn\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061089 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-system-cni-dir\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061103 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-run-k8s-cni-cncf-io\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061115 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1443215c-f357-4e92-a03d-ee65a659522d-multus-daemon-config\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061140 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ecd9d42a-2391-4e4e-b2e1-1f3252628680-iptables-alerter-script\") pod \"iptables-alerter-pqc4p\" (UID: \"ecd9d42a-2391-4e4e-b2e1-1f3252628680\") " pod="openshift-network-operator/iptables-alerter-pqc4p" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061142 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-run-ovn\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061159 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-system-cni-dir\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061165 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/28f013b7-2bc7-4ed9-9284-6ac5f5922566-konnectivity-ca\") pod \"konnectivity-agent-chsnd\" (UID: \"28f013b7-2bc7-4ed9-9284-6ac5f5922566\") " pod="kube-system/konnectivity-agent-chsnd" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061212 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-sysctl-d\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061238 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rdh5\" (UniqueName: \"kubernetes.io/projected/eaeccbf3-20f1-42cd-b6df-814ce481cb63-kube-api-access-4rdh5\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061274 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-ovnkube-script-lib\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061297 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/28f013b7-2bc7-4ed9-9284-6ac5f5922566-agent-certs\") pod \"konnectivity-agent-chsnd\" (UID: \"28f013b7-2bc7-4ed9-9284-6ac5f5922566\") " pod="kube-system/konnectivity-agent-chsnd" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061320 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-lib-modules\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061343 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-ovn-node-metrics-cert\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061353 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-sysctl-d\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061365 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-run-netns\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.064677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061405 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/395c28b2-8a48-403f-81b0-51e8e602646d-serviceca\") pod \"node-ca-ssg6q\" (UID: \"395c28b2-8a48-403f-81b0-51e8e602646d\") " pod="openshift-image-registry/node-ca-ssg6q" Jul 07 15:01:13.065366 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061422 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1443215c-f357-4e92-a03d-ee65a659522d-host-run-netns\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.065366 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061436 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-socket-dir\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.065366 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061542 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-lib-modules\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.065366 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061585 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eaeccbf3-20f1-42cd-b6df-814ce481cb63-socket-dir\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.065366 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061649 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1443215c-f357-4e92-a03d-ee65a659522d-multus-daemon-config\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.065366 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.061679 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ecd9d42a-2391-4e4e-b2e1-1f3252628680-iptables-alerter-script\") pod \"iptables-alerter-pqc4p\" (UID: \"ecd9d42a-2391-4e4e-b2e1-1f3252628680\") " pod="openshift-network-operator/iptables-alerter-pqc4p" Jul 07 15:01:13.065366 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.062101 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-ovnkube-script-lib\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.065366 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.064829 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-ovn-node-metrics-cert\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.065366 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.064888 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-etc-tuned\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.066483 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.066465 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-tmp\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.073839 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.073811 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prbmp\" (UniqueName: \"kubernetes.io/projected/ddfa38fd-1217-4757-a9d3-b4c0bd85211c-kube-api-access-prbmp\") pod \"tuned-9z6z7\" (UID: \"ddfa38fd-1217-4757-a9d3-b4c0bd85211c\") " pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.075448 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.075402 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrzgk\" (UniqueName: \"kubernetes.io/projected/3dece03e-eb3a-4661-855f-f37fca6d1923-kube-api-access-zrzgk\") pod \"network-metrics-daemon-kfp7w\" (UID: \"3dece03e-eb3a-4661-855f-f37fca6d1923\") " pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:13.076257 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.076236 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8n2n\" (UniqueName: \"kubernetes.io/projected/ecd9d42a-2391-4e4e-b2e1-1f3252628680-kube-api-access-f8n2n\") pod \"iptables-alerter-pqc4p\" (UID: \"ecd9d42a-2391-4e4e-b2e1-1f3252628680\") " pod="openshift-network-operator/iptables-alerter-pqc4p" Jul 07 15:01:13.076625 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.076596 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2r72\" (UniqueName: \"kubernetes.io/projected/2def56c2-e1cd-4af7-8d6b-00e56dc39e1e-kube-api-access-h2r72\") pod \"ovnkube-node-snw9m\" (UID: \"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e\") " pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.077086 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.077066 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl9hx\" (UniqueName: \"kubernetes.io/projected/1443215c-f357-4e92-a03d-ee65a659522d-kube-api-access-cl9hx\") pod \"multus-kbdqw\" (UID: \"1443215c-f357-4e92-a03d-ee65a659522d\") " pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.077874 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.077831 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rdh5\" (UniqueName: \"kubernetes.io/projected/eaeccbf3-20f1-42cd-b6df-814ce481cb63-kube-api-access-4rdh5\") pod \"aws-ebs-csi-driver-node-vfjr9\" (UID: \"eaeccbf3-20f1-42cd-b6df-814ce481cb63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.079099 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.079077 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x2m7\" (UniqueName: \"kubernetes.io/projected/15be5c3f-36e0-4a8d-be13-09a8cc5e73fe-kube-api-access-2x2m7\") pod \"multus-additional-cni-plugins-qz6xz\" (UID: \"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe\") " pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.103332 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.103283 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-205.ec2.internal" event={"ID":"0a252f3cefa1fdcdb223e8cd15c86c48","Type":"ContainerStarted","Data":"3fd7451bf1c689ae8d1c5af103d1386c22ca54874a10f5096f0a7ff3990441ff"} Jul 07 15:01:13.104974 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.104928 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal" event={"ID":"6cc11541a599eef8412e2d8adc913c1e","Type":"ContainerStarted","Data":"d671cd1bb3219c9ce8234681888ddb12a76f096f5f09663c7b97b25d5bbbcb67"} Jul 07 15:01:13.161992 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.161923 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae612123-dc47-43fd-b0aa-93d175975cd6-kubelet-config\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:13.161992 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.161990 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d-hosts-file\") pod \"node-resolver-6wgcw\" (UID: \"bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d\") " pod="openshift-dns/node-resolver-6wgcw" Jul 07 15:01:13.162197 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162023 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae612123-dc47-43fd-b0aa-93d175975cd6-kubelet-config\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:13.162197 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162035 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/28f013b7-2bc7-4ed9-9284-6ac5f5922566-konnectivity-ca\") pod \"konnectivity-agent-chsnd\" (UID: \"28f013b7-2bc7-4ed9-9284-6ac5f5922566\") " pod="kube-system/konnectivity-agent-chsnd" Jul 07 15:01:13.162197 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162161 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d-hosts-file\") pod \"node-resolver-6wgcw\" (UID: \"bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d\") " pod="openshift-dns/node-resolver-6wgcw" Jul 07 15:01:13.162352 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162219 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/28f013b7-2bc7-4ed9-9284-6ac5f5922566-agent-certs\") pod \"konnectivity-agent-chsnd\" (UID: \"28f013b7-2bc7-4ed9-9284-6ac5f5922566\") " pod="kube-system/konnectivity-agent-chsnd" Jul 07 15:01:13.162352 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162250 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/395c28b2-8a48-403f-81b0-51e8e602646d-serviceca\") pod \"node-ca-ssg6q\" (UID: \"395c28b2-8a48-403f-81b0-51e8e602646d\") " pod="openshift-image-registry/node-ca-ssg6q" Jul 07 15:01:13.162352 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162297 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/395c28b2-8a48-403f-81b0-51e8e602646d-host\") pod \"node-ca-ssg6q\" (UID: \"395c28b2-8a48-403f-81b0-51e8e602646d\") " pod="openshift-image-registry/node-ca-ssg6q" Jul 07 15:01:13.162352 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb9ql\" (UniqueName: \"kubernetes.io/projected/bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d-kube-api-access-rb9ql\") pod \"node-resolver-6wgcw\" (UID: \"bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d\") " pod="openshift-dns/node-resolver-6wgcw" Jul 07 15:01:13.162544 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162387 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d-tmp-dir\") pod \"node-resolver-6wgcw\" (UID: \"bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d\") " pod="openshift-dns/node-resolver-6wgcw" Jul 07 15:01:13.162544 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162414 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vgkv\" (UniqueName: \"kubernetes.io/projected/395c28b2-8a48-403f-81b0-51e8e602646d-kube-api-access-5vgkv\") pod \"node-ca-ssg6q\" (UID: \"395c28b2-8a48-403f-81b0-51e8e602646d\") " pod="openshift-image-registry/node-ca-ssg6q" Jul 07 15:01:13.162544 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162443 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gkgs\" (UniqueName: \"kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs\") pod \"network-check-target-j52v9\" (UID: \"8b6865e3-214f-4260-87bd-66c295f1e763\") " pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:13.162544 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162470 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae612123-dc47-43fd-b0aa-93d175975cd6-dbus\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:13.162544 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162495 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:13.162825 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162561 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/28f013b7-2bc7-4ed9-9284-6ac5f5922566-konnectivity-ca\") pod \"konnectivity-agent-chsnd\" (UID: \"28f013b7-2bc7-4ed9-9284-6ac5f5922566\") " pod="kube-system/konnectivity-agent-chsnd" Jul 07 15:01:13.162825 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.162604 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:13.162825 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.162657 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret podName:ae612123-dc47-43fd-b0aa-93d175975cd6 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:13.662641043 +0000 UTC m=+3.088029424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret") pod "global-pull-secret-syncer-66m5k" (UID: "ae612123-dc47-43fd-b0aa-93d175975cd6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:13.162825 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162661 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/395c28b2-8a48-403f-81b0-51e8e602646d-host\") pod \"node-ca-ssg6q\" (UID: \"395c28b2-8a48-403f-81b0-51e8e602646d\") " pod="openshift-image-registry/node-ca-ssg6q" Jul 07 15:01:13.162825 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162781 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae612123-dc47-43fd-b0aa-93d175975cd6-dbus\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:13.163117 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.162990 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/395c28b2-8a48-403f-81b0-51e8e602646d-serviceca\") pod \"node-ca-ssg6q\" (UID: \"395c28b2-8a48-403f-81b0-51e8e602646d\") " pod="openshift-image-registry/node-ca-ssg6q" Jul 07 15:01:13.163117 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.163096 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d-tmp-dir\") pod \"node-resolver-6wgcw\" (UID: \"bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d\") " pod="openshift-dns/node-resolver-6wgcw" Jul 07 15:01:13.165300 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.165276 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/28f013b7-2bc7-4ed9-9284-6ac5f5922566-agent-certs\") pod \"konnectivity-agent-chsnd\" (UID: \"28f013b7-2bc7-4ed9-9284-6ac5f5922566\") " pod="kube-system/konnectivity-agent-chsnd" Jul 07 15:01:13.173670 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.173643 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vgkv\" (UniqueName: \"kubernetes.io/projected/395c28b2-8a48-403f-81b0-51e8e602646d-kube-api-access-5vgkv\") pod \"node-ca-ssg6q\" (UID: \"395c28b2-8a48-403f-81b0-51e8e602646d\") " pod="openshift-image-registry/node-ca-ssg6q" Jul 07 15:01:13.176839 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.176821 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:01:13.176955 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.176845 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:01:13.176955 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.176859 2565 projected.go:196] Error preparing data for projected volume kube-api-access-9gkgs for pod openshift-network-diagnostics/network-check-target-j52v9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:13.176955 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.176919 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs podName:8b6865e3-214f-4260-87bd-66c295f1e763 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:13.676902416 +0000 UTC m=+3.102290788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9gkgs" (UniqueName: "kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs") pod "network-check-target-j52v9" (UID: "8b6865e3-214f-4260-87bd-66c295f1e763") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:13.178878 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.178847 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb9ql\" (UniqueName: \"kubernetes.io/projected/bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d-kube-api-access-rb9ql\") pod \"node-resolver-6wgcw\" (UID: \"bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d\") " pod="openshift-dns/node-resolver-6wgcw" Jul 07 15:01:13.249376 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.249307 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" Jul 07 15:01:13.262292 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.262255 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qz6xz" Jul 07 15:01:13.270892 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.270865 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" Jul 07 15:01:13.278102 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.278082 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:13.284862 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.284843 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kbdqw" Jul 07 15:01:13.293565 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.293543 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pqc4p" Jul 07 15:01:13.302361 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.302343 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-chsnd" Jul 07 15:01:13.310100 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.310076 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6wgcw" Jul 07 15:01:13.316999 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.316978 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ssg6q" Jul 07 15:01:13.565282 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.565193 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs\") pod \"network-metrics-daemon-kfp7w\" (UID: \"3dece03e-eb3a-4661-855f-f37fca6d1923\") " pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:13.565433 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.565363 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:13.565498 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.565433 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs podName:3dece03e-eb3a-4661-855f-f37fca6d1923 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:14.565413802 +0000 UTC m=+3.990802168 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs") pod "network-metrics-daemon-kfp7w" (UID: "3dece03e-eb3a-4661-855f-f37fca6d1923") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:13.666581 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.666553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:13.666747 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.666695 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:13.666801 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.666762 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret podName:ae612123-dc47-43fd-b0aa-93d175975cd6 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:14.666742987 +0000 UTC m=+4.092131372 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret") pod "global-pull-secret-syncer-66m5k" (UID: "ae612123-dc47-43fd-b0aa-93d175975cd6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:13.767740 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.767706 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gkgs\" (UniqueName: \"kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs\") pod \"network-check-target-j52v9\" (UID: \"8b6865e3-214f-4260-87bd-66c295f1e763\") " pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:13.767934 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.767840 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:01:13.767934 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.767866 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:01:13.767934 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.767881 2565 projected.go:196] Error preparing data for projected volume kube-api-access-9gkgs for pod openshift-network-diagnostics/network-check-target-j52v9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:13.768121 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:13.767958 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs podName:8b6865e3-214f-4260-87bd-66c295f1e763 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:14.767922968 +0000 UTC m=+4.193311336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gkgs" (UniqueName: "kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs") pod "network-check-target-j52v9" (UID: "8b6865e3-214f-4260-87bd-66c295f1e763") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:13.992582 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.992438 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:56:12 +0000 UTC" deadline="2028-01-08 21:29:17.82471697 +0000 UTC" Jul 07 15:01:13.992582 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:13.992528 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13206h28m3.832198611s" Jul 07 15:01:14.015430 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:14.015392 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:01:14.098077 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:14.098047 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:14.098254 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:14.098202 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:14.261774 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:14.261745 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf1e3894_e18e_4ea6_bb7b_b7ab0338c85d.slice/crio-adfbe9a442e8dad829c67fb1992f3a9f3db5c307fbac1a2ba695232299733003 WatchSource:0}: Error finding container adfbe9a442e8dad829c67fb1992f3a9f3db5c307fbac1a2ba695232299733003: Status 404 returned error can't find the container with id adfbe9a442e8dad829c67fb1992f3a9f3db5c307fbac1a2ba695232299733003 Jul 07 15:01:14.264759 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:14.264728 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15be5c3f_36e0_4a8d_be13_09a8cc5e73fe.slice/crio-843c11029949648b2dbb42013e90521a77f2dc1c7bfa07360f8857b6ca336bc4 WatchSource:0}: Error finding container 843c11029949648b2dbb42013e90521a77f2dc1c7bfa07360f8857b6ca336bc4: Status 404 returned error can't find the container with id 843c11029949648b2dbb42013e90521a77f2dc1c7bfa07360f8857b6ca336bc4 Jul 07 15:01:14.266957 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:14.266387 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2def56c2_e1cd_4af7_8d6b_00e56dc39e1e.slice/crio-3c52df741da9dbd2e0b9f2a59fd03d2762f1e142ba3adee25119d7d39e05b8bb WatchSource:0}: Error finding container 3c52df741da9dbd2e0b9f2a59fd03d2762f1e142ba3adee25119d7d39e05b8bb: Status 404 returned error can't find the container with id 3c52df741da9dbd2e0b9f2a59fd03d2762f1e142ba3adee25119d7d39e05b8bb Jul 07 15:01:14.267487 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:14.267458 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f013b7_2bc7_4ed9_9284_6ac5f5922566.slice/crio-10820106f3a92e5f26d76c50283f9cb3b4fcfc68048983b1c8923232f7476133 WatchSource:0}: Error finding container 10820106f3a92e5f26d76c50283f9cb3b4fcfc68048983b1c8923232f7476133: Status 404 returned error can't find the container with id 10820106f3a92e5f26d76c50283f9cb3b4fcfc68048983b1c8923232f7476133 Jul 07 15:01:14.269662 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:14.269505 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecd9d42a_2391_4e4e_b2e1_1f3252628680.slice/crio-784fa922d75092ed4874f76a6e1f81bd05e14c5e8f5ddfa0b1b8877a5695ab33 WatchSource:0}: Error finding container 784fa922d75092ed4874f76a6e1f81bd05e14c5e8f5ddfa0b1b8877a5695ab33: Status 404 returned error can't find the container with id 784fa922d75092ed4874f76a6e1f81bd05e14c5e8f5ddfa0b1b8877a5695ab33 Jul 07 15:01:14.270473 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:14.270452 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaeccbf3_20f1_42cd_b6df_814ce481cb63.slice/crio-a2cec0d29a9641a2d33c9ac04a39a52cfbcd5b80846d708b36c7f09471bacbde WatchSource:0}: Error finding container a2cec0d29a9641a2d33c9ac04a39a52cfbcd5b80846d708b36c7f09471bacbde: Status 404 returned error can't find the container with id a2cec0d29a9641a2d33c9ac04a39a52cfbcd5b80846d708b36c7f09471bacbde Jul 07 15:01:14.271071 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:14.271050 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1443215c_f357_4e92_a03d_ee65a659522d.slice/crio-5415c77ec882185422bfdbff79a781e739e29120f89bf137bca682e06a1b3614 WatchSource:0}: Error finding container 5415c77ec882185422bfdbff79a781e739e29120f89bf137bca682e06a1b3614: Status 404 returned error can't find the container with id 5415c77ec882185422bfdbff79a781e739e29120f89bf137bca682e06a1b3614 Jul 07 15:01:14.273145 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:14.273123 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddfa38fd_1217_4757_a9d3_b4c0bd85211c.slice/crio-58d7d68d8d3b222ecb3afed69ef0f4c182c71618f35c8f38a068a114a5d40a5a WatchSource:0}: Error finding container 58d7d68d8d3b222ecb3afed69ef0f4c182c71618f35c8f38a068a114a5d40a5a: Status 404 returned error can't find the container with id 58d7d68d8d3b222ecb3afed69ef0f4c182c71618f35c8f38a068a114a5d40a5a Jul 07 15:01:14.275087 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:14.274598 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395c28b2_8a48_403f_81b0_51e8e602646d.slice/crio-3f73e8bb783a882880b0e7f8fb2cf022e8007ead75d3b9448ad535aafac2b922 WatchSource:0}: Error finding container 3f73e8bb783a882880b0e7f8fb2cf022e8007ead75d3b9448ad535aafac2b922: Status 404 returned error can't find the container with id 3f73e8bb783a882880b0e7f8fb2cf022e8007ead75d3b9448ad535aafac2b922 Jul 07 15:01:14.572698 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:14.572484 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs\") pod \"network-metrics-daemon-kfp7w\" (UID: \"3dece03e-eb3a-4661-855f-f37fca6d1923\") " pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:14.572790 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:14.572604 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:14.572790 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:14.572771 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs podName:3dece03e-eb3a-4661-855f-f37fca6d1923 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:16.572756287 +0000 UTC m=+5.998144655 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs") pod "network-metrics-daemon-kfp7w" (UID: "3dece03e-eb3a-4661-855f-f37fca6d1923") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:14.673667 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:14.673632 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:14.673820 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:14.673768 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:14.673881 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:14.673837 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret podName:ae612123-dc47-43fd-b0aa-93d175975cd6 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:16.6738183 +0000 UTC m=+6.099206683 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret") pod "global-pull-secret-syncer-66m5k" (UID: "ae612123-dc47-43fd-b0aa-93d175975cd6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:14.775070 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:14.774923 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gkgs\" (UniqueName: \"kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs\") pod \"network-check-target-j52v9\" (UID: \"8b6865e3-214f-4260-87bd-66c295f1e763\") " pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:14.775228 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:14.775085 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:01:14.775228 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:14.775108 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:01:14.775228 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:14.775121 2565 projected.go:196] Error preparing data for projected volume kube-api-access-9gkgs for pod openshift-network-diagnostics/network-check-target-j52v9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:14.775228 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:14.775182 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs podName:8b6865e3-214f-4260-87bd-66c295f1e763 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:16.775163619 +0000 UTC m=+6.200551988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gkgs" (UniqueName: "kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs") pod "network-check-target-j52v9" (UID: "8b6865e3-214f-4260-87bd-66c295f1e763") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:14.992988 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:14.992926 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:56:12 +0000 UTC" deadline="2027-12-09 18:34:53.247854647 +0000 UTC" Jul 07 15:01:14.992988 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:14.992987 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12483h33m38.254872347s" Jul 07 15:01:15.102164 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.101465 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:15.102164 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:15.101583 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:15.102164 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.101983 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:15.102164 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:15.102074 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:15.123043 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.122136 2565 generic.go:356] "Generic (PLEG): container finished" podID="6cc11541a599eef8412e2d8adc913c1e" containerID="9598795e09b2f9c6229016fac17631f47d338ae88c6c234a43c1b494d84e77a2" exitCode=0 Jul 07 15:01:15.123043 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.122277 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal" event={"ID":"6cc11541a599eef8412e2d8adc913c1e","Type":"ContainerDied","Data":"9598795e09b2f9c6229016fac17631f47d338ae88c6c234a43c1b494d84e77a2"} Jul 07 15:01:15.131431 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.131357 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ssg6q" event={"ID":"395c28b2-8a48-403f-81b0-51e8e602646d","Type":"ContainerStarted","Data":"3f73e8bb783a882880b0e7f8fb2cf022e8007ead75d3b9448ad535aafac2b922"} Jul 07 15:01:15.144968 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.144925 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kbdqw" event={"ID":"1443215c-f357-4e92-a03d-ee65a659522d","Type":"ContainerStarted","Data":"5415c77ec882185422bfdbff79a781e739e29120f89bf137bca682e06a1b3614"} Jul 07 15:01:15.167190 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.167156 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" event={"ID":"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e","Type":"ContainerStarted","Data":"3c52df741da9dbd2e0b9f2a59fd03d2762f1e142ba3adee25119d7d39e05b8bb"} Jul 07 15:01:15.173519 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.173488 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qz6xz" event={"ID":"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe","Type":"ContainerStarted","Data":"843c11029949648b2dbb42013e90521a77f2dc1c7bfa07360f8857b6ca336bc4"} Jul 07 15:01:15.187997 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.187965 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6wgcw" event={"ID":"bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d","Type":"ContainerStarted","Data":"adfbe9a442e8dad829c67fb1992f3a9f3db5c307fbac1a2ba695232299733003"} Jul 07 15:01:15.196486 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.196452 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-205.ec2.internal" event={"ID":"0a252f3cefa1fdcdb223e8cd15c86c48","Type":"ContainerStarted","Data":"af69f5623331865013a1e4e340df28ca2f878d8d96a0eb1303844363ed991003"} Jul 07 15:01:15.214917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.214851 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-205.ec2.internal" podStartSLOduration=3.214833687 podStartE2EDuration="3.214833687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:15.214027193 +0000 UTC m=+4.639415577" watchObservedRunningTime="2026-07-07 15:01:15.214833687 +0000 UTC m=+4.640222075" Jul 07 15:01:15.215821 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.215795 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" event={"ID":"ddfa38fd-1217-4757-a9d3-b4c0bd85211c","Type":"ContainerStarted","Data":"58d7d68d8d3b222ecb3afed69ef0f4c182c71618f35c8f38a068a114a5d40a5a"} Jul 07 15:01:15.247986 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.247850 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" event={"ID":"eaeccbf3-20f1-42cd-b6df-814ce481cb63","Type":"ContainerStarted","Data":"a2cec0d29a9641a2d33c9ac04a39a52cfbcd5b80846d708b36c7f09471bacbde"} Jul 07 15:01:15.252832 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.252802 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-chsnd" event={"ID":"28f013b7-2bc7-4ed9-9284-6ac5f5922566","Type":"ContainerStarted","Data":"10820106f3a92e5f26d76c50283f9cb3b4fcfc68048983b1c8923232f7476133"} Jul 07 15:01:15.265223 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:15.264854 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pqc4p" event={"ID":"ecd9d42a-2391-4e4e-b2e1-1f3252628680","Type":"ContainerStarted","Data":"784fa922d75092ed4874f76a6e1f81bd05e14c5e8f5ddfa0b1b8877a5695ab33"} Jul 07 15:01:16.098012 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:16.097977 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:16.098505 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:16.098130 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:16.285110 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:16.285065 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal" event={"ID":"6cc11541a599eef8412e2d8adc913c1e","Type":"ContainerStarted","Data":"dfe89e141026fd415e9c37781f789b3c569fae462ecfa45ee2e40026efc46a38"} Jul 07 15:01:16.301463 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:16.301407 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-205.ec2.internal" podStartSLOduration=4.301385723 podStartE2EDuration="4.301385723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:16.300638461 +0000 UTC m=+5.726026850" watchObservedRunningTime="2026-07-07 15:01:16.301385723 +0000 UTC m=+5.726774115" Jul 07 15:01:16.589404 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:16.588614 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs\") pod \"network-metrics-daemon-kfp7w\" (UID: \"3dece03e-eb3a-4661-855f-f37fca6d1923\") " pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:16.589404 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:16.588879 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:16.589404 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:16.588958 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs podName:3dece03e-eb3a-4661-855f-f37fca6d1923 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:20.588928382 +0000 UTC m=+10.014316750 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs") pod "network-metrics-daemon-kfp7w" (UID: "3dece03e-eb3a-4661-855f-f37fca6d1923") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:16.689225 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:16.689191 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:16.689383 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:16.689362 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:16.689455 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:16.689437 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret podName:ae612123-dc47-43fd-b0aa-93d175975cd6 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:20.689416152 +0000 UTC m=+10.114804538 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret") pod "global-pull-secret-syncer-66m5k" (UID: "ae612123-dc47-43fd-b0aa-93d175975cd6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:16.789740 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:16.789705 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gkgs\" (UniqueName: \"kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs\") pod \"network-check-target-j52v9\" (UID: \"8b6865e3-214f-4260-87bd-66c295f1e763\") " pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:16.789912 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:16.789893 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:01:16.789912 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:16.789910 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:01:16.790064 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:16.789922 2565 projected.go:196] Error preparing data for projected volume kube-api-access-9gkgs for pod openshift-network-diagnostics/network-check-target-j52v9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:16.790064 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:16.789992 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs podName:8b6865e3-214f-4260-87bd-66c295f1e763 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:20.789974258 +0000 UTC m=+10.215362644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gkgs" (UniqueName: "kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs") pod "network-check-target-j52v9" (UID: "8b6865e3-214f-4260-87bd-66c295f1e763") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:17.099488 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:17.099045 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:17.099488 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:17.099168 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:17.099488 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:17.099220 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:17.099488 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:17.099351 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:18.098353 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:18.098322 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:18.098489 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:18.098472 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:19.098967 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:19.098275 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:19.098967 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:19.098409 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:19.098967 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:19.098791 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:19.098967 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:19.098889 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:20.098314 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:20.098273 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:20.098509 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:20.098421 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:20.628071 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:20.628032 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs\") pod \"network-metrics-daemon-kfp7w\" (UID: \"3dece03e-eb3a-4661-855f-f37fca6d1923\") " pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:20.628544 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:20.628218 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:20.628544 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:20.628294 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs podName:3dece03e-eb3a-4661-855f-f37fca6d1923 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:28.628273128 +0000 UTC m=+18.053661520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs") pod "network-metrics-daemon-kfp7w" (UID: "3dece03e-eb3a-4661-855f-f37fca6d1923") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:20.728671 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:20.728629 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:20.728856 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:20.728771 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:20.728856 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:20.728833 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret podName:ae612123-dc47-43fd-b0aa-93d175975cd6 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:28.728816098 +0000 UTC m=+18.154204483 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret") pod "global-pull-secret-syncer-66m5k" (UID: "ae612123-dc47-43fd-b0aa-93d175975cd6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:20.829681 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:20.829640 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gkgs\" (UniqueName: \"kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs\") pod \"network-check-target-j52v9\" (UID: \"8b6865e3-214f-4260-87bd-66c295f1e763\") " pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:20.829889 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:20.829835 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:01:20.829889 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:20.829859 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:01:20.829889 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:20.829873 2565 projected.go:196] Error preparing data for projected volume kube-api-access-9gkgs for pod openshift-network-diagnostics/network-check-target-j52v9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:20.830077 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:20.829951 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs podName:8b6865e3-214f-4260-87bd-66c295f1e763 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:28.829917949 +0000 UTC m=+18.255306332 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gkgs" (UniqueName: "kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs") pod "network-check-target-j52v9" (UID: "8b6865e3-214f-4260-87bd-66c295f1e763") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:21.099826 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:21.099379 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:21.099826 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:21.099499 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:21.099826 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:21.099551 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:21.099826 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:21.099664 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:22.098401 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:22.098220 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:22.098401 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:22.098364 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:23.097767 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:23.097732 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:23.098004 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:23.097869 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:23.098004 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:23.097731 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:23.098004 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:23.097985 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:24.098806 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:24.098758 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:24.099327 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:24.098914 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:25.098514 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:25.098427 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:25.098514 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:25.098469 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:25.098711 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:25.098570 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:25.098711 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:25.098696 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:26.098298 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:26.098267 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:26.098748 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:26.098401 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:27.098387 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:27.098343 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:27.098387 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:27.098367 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:27.098924 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:27.098516 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:27.099010 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:27.098983 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:28.098091 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:28.098057 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:28.098300 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:28.098184 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:28.691672 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:28.691639 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs\") pod \"network-metrics-daemon-kfp7w\" (UID: \"3dece03e-eb3a-4661-855f-f37fca6d1923\") " pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:28.692173 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:28.691800 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:28.692173 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:28.691875 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs podName:3dece03e-eb3a-4661-855f-f37fca6d1923 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:44.691852837 +0000 UTC m=+34.117241207 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs") pod "network-metrics-daemon-kfp7w" (UID: "3dece03e-eb3a-4661-855f-f37fca6d1923") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:28.792567 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:28.792532 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:28.792719 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:28.792657 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:28.792719 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:28.792710 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret podName:ae612123-dc47-43fd-b0aa-93d175975cd6 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:44.792696904 +0000 UTC m=+34.218085270 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret") pod "global-pull-secret-syncer-66m5k" (UID: "ae612123-dc47-43fd-b0aa-93d175975cd6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:28.893985 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:28.893931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gkgs\" (UniqueName: \"kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs\") pod \"network-check-target-j52v9\" (UID: \"8b6865e3-214f-4260-87bd-66c295f1e763\") " pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:28.894158 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:28.894132 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:01:28.894158 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:28.894156 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:01:28.894253 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:28.894169 2565 projected.go:196] Error preparing data for projected volume kube-api-access-9gkgs for pod openshift-network-diagnostics/network-check-target-j52v9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:28.894253 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:28.894238 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs podName:8b6865e3-214f-4260-87bd-66c295f1e763 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:44.894218664 +0000 UTC m=+34.319607111 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gkgs" (UniqueName: "kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs") pod "network-check-target-j52v9" (UID: "8b6865e3-214f-4260-87bd-66c295f1e763") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:29.098321 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:29.098238 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:29.098479 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:29.098238 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:29.098479 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:29.098378 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:29.098479 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:29.098438 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:30.098509 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:30.098474 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:30.099005 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:30.098599 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:31.098897 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:31.098845 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:31.099377 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:31.098974 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:31.099377 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:31.099361 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:31.099490 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:31.099464 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:32.098791 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:32.098750 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:32.098973 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:32.098875 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:33.098778 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:33.098750 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:33.098778 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:33.098782 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:33.098992 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:33.098915 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:33.098992 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:33.098970 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:33.333243 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:33.333016 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6wgcw" event={"ID":"bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d","Type":"ContainerStarted","Data":"f089654fe63f2ceacf6baa27aba15f4170c3a316b3653dcfc0be16301e34c889"} Jul 07 15:01:34.098548 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.098313 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:34.098744 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:34.098648 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:34.336723 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.336683 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" event={"ID":"ddfa38fd-1217-4757-a9d3-b4c0bd85211c","Type":"ContainerStarted","Data":"bb0ca10ad50de306da51df061b0728a74108045ffcc5f5c375326260fe15be19"} Jul 07 15:01:34.338314 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.338288 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" event={"ID":"eaeccbf3-20f1-42cd-b6df-814ce481cb63","Type":"ContainerStarted","Data":"a964c126c56c7a5263826e0abed4135c3c0e0211a086a2789ba64a39984b56e4"} Jul 07 15:01:34.339771 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.339743 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-chsnd" event={"ID":"28f013b7-2bc7-4ed9-9284-6ac5f5922566","Type":"ContainerStarted","Data":"2a9393b9530353512f9c603c142f60e0961e362c55e727bb5e017aa2182f6d33"} Jul 07 15:01:34.341275 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.341197 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ssg6q" event={"ID":"395c28b2-8a48-403f-81b0-51e8e602646d","Type":"ContainerStarted","Data":"980aca5bcfbdc6fdea3f45a743e5acf47bf00004843d64f3947a16762cbe68e4"} Jul 07 15:01:34.342796 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.342767 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kbdqw" event={"ID":"1443215c-f357-4e92-a03d-ee65a659522d","Type":"ContainerStarted","Data":"52abf05cb3023c2ea38e3c9229d4dfef4cbeedb7f56a51ca4de46b62fe8aa3de"} Jul 07 15:01:34.345776 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.345761 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:01:34.346127 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.346108 2565 generic.go:356] "Generic (PLEG): container finished" podID="2def56c2-e1cd-4af7-8d6b-00e56dc39e1e" containerID="f61de84f7a6d1e602f1757e37d00b4e60bad2b2ad07551cdb92d716e21c3445c" exitCode=1 Jul 07 15:01:34.346222 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.346172 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" event={"ID":"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e","Type":"ContainerStarted","Data":"ae533e94a65dbe838c1bb9e3fc2059d469bc8544d3c03b0aa5fa84ccf35ca923"} Jul 07 15:01:34.346266 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.346240 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" event={"ID":"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e","Type":"ContainerStarted","Data":"73c5685ff756a0567f371ab38a0aa678fd2fada5a9a731aeb87ab7e84a1a40f3"} Jul 07 15:01:34.346314 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.346283 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" event={"ID":"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e","Type":"ContainerStarted","Data":"61bdd036e91e76c9500630f114ddd61e5778a64f47aa8fcc4663866320101df2"} Jul 07 15:01:34.346356 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.346329 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" event={"ID":"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e","Type":"ContainerStarted","Data":"93bf72647b8fa56b87f8dc79d11790f98db5bf3c5792120463038cc440f5597a"} Jul 07 15:01:34.346403 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.346375 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" event={"ID":"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e","Type":"ContainerDied","Data":"f61de84f7a6d1e602f1757e37d00b4e60bad2b2ad07551cdb92d716e21c3445c"} Jul 07 15:01:34.346454 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.346419 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" event={"ID":"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e","Type":"ContainerStarted","Data":"0f616a64ee5acc019fea49ef0d229a92c6602009abcb371f4d75d28cadb5cf4e"} Jul 07 15:01:34.347695 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.347672 2565 generic.go:356] "Generic (PLEG): container finished" podID="15be5c3f-36e0-4a8d-be13-09a8cc5e73fe" containerID="c5bfb533754e666845a0eb9c90944a2a2f751d283ed2dbbed9db30c215f56777" exitCode=0 Jul 07 15:01:34.347787 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.347725 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qz6xz" event={"ID":"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe","Type":"ContainerDied","Data":"c5bfb533754e666845a0eb9c90944a2a2f751d283ed2dbbed9db30c215f56777"} Jul 07 15:01:34.364351 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.364297 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9z6z7" podStartSLOduration=4.458405678 podStartE2EDuration="23.364281055s" totalImagesPullingTime="18.905875377s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:34.363880549 +0000 UTC m=+23.789268937" watchObservedRunningTime="2026-07-07 15:01:34.364281055 +0000 UTC m=+23.789669450" Jul 07 15:01:34.420285 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.420220 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kbdqw" podStartSLOduration=4.470302344 podStartE2EDuration="23.420203821s" totalImagesPullingTime="18.949901477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:34.419236589 +0000 UTC m=+23.844624978" watchObservedRunningTime="2026-07-07 15:01:34.420203821 +0000 UTC m=+23.845592224" Jul 07 15:01:34.462737 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.462659 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6wgcw" podStartSLOduration=4.833147621 podStartE2EDuration="23.462642458s" totalImagesPullingTime="18.629494837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:34.46237474 +0000 UTC m=+23.887763128" watchObservedRunningTime="2026-07-07 15:01:34.462642458 +0000 UTC m=+23.888030847" Jul 07 15:01:34.463008 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.462983 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-chsnd" podStartSLOduration=4.839381025 podStartE2EDuration="23.462976762s" totalImagesPullingTime="18.623595737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:34.443005209 +0000 UTC m=+23.868393596" watchObservedRunningTime="2026-07-07 15:01:34.462976762 +0000 UTC m=+23.888365149" Jul 07 15:01:34.485087 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.485048 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ssg6q" podStartSLOduration=4.603020762 podStartE2EDuration="23.485040772s" totalImagesPullingTime="18.88202001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:34.484669957 +0000 UTC m=+23.910058325" watchObservedRunningTime="2026-07-07 15:01:34.485040772 +0000 UTC m=+23.910429160" Jul 07 15:01:34.995856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:34.995836 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 15:01:35.016114 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:35.015908 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T15:01:34.995852335Z","UUID":"a09b7dbc-8d10-4e75-b837-946135f6937e","Handler":null,"Name":"","Endpoint":""} Jul 07 15:01:35.017476 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:35.017451 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 15:01:35.017576 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:35.017487 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 15:01:35.098285 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:35.098253 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:35.098439 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:35.098289 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:35.098439 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:35.098376 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:35.098514 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:35.098493 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:35.352218 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:35.352183 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" event={"ID":"eaeccbf3-20f1-42cd-b6df-814ce481cb63","Type":"ContainerStarted","Data":"8148fecab1d1b026dabe89f6bca5f86167361ef65c32ed1e6b2bcefea359f6a9"} Jul 07 15:01:35.353800 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:35.353771 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pqc4p" event={"ID":"ecd9d42a-2391-4e4e-b2e1-1f3252628680","Type":"ContainerStarted","Data":"1e7fe938af2dcf6203b874d27383209bb66664223881f308417b3c758b64bbbd"} Jul 07 15:01:35.380330 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:35.380285 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pqc4p" podStartSLOduration=5.467259938 podStartE2EDuration="24.380265416s" totalImagesPullingTime="18.913005478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:35.379883684 +0000 UTC m=+24.805272073" watchObservedRunningTime="2026-07-07 15:01:35.380265416 +0000 UTC m=+24.805653806" Jul 07 15:01:36.098121 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:36.098096 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:36.098285 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:36.098209 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:37.098845 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:37.098642 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:37.099368 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:37.098642 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:37.099368 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:37.098971 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:37.099368 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:37.099013 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:37.359744 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:37.359652 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" event={"ID":"eaeccbf3-20f1-42cd-b6df-814ce481cb63","Type":"ContainerStarted","Data":"50f0e81ccaabfe69cfc265da4133be87b0f201d92afd0a3b19d8049ed5a2deff"} Jul 07 15:01:37.362600 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:37.362580 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:01:37.362984 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:37.362966 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" event={"ID":"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e","Type":"ContainerStarted","Data":"b6a011adbeb2536b046ccd7741fc764b0d22453a03689b8b458694f1c34a7190"} Jul 07 15:01:37.385466 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:37.385403 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfjr9" podStartSLOduration=4.50903237 podStartE2EDuration="26.385374133s" totalImagesPullingTime="21.876341763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:01:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:37.384162584 +0000 UTC m=+26.809550974" watchObservedRunningTime="2026-07-07 15:01:37.385374133 +0000 UTC m=+26.810762522" Jul 07 15:01:38.098814 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:38.098786 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:38.099014 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:38.098914 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:38.136026 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:38.135989 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-chsnd" Jul 07 15:01:38.136546 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:38.136524 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-chsnd" Jul 07 15:01:39.098896 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:39.098859 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:39.099082 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:39.098859 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:39.099082 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:39.099008 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:39.099082 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:39.099027 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:39.784845 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:39.784810 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-chsnd" Jul 07 15:01:39.785048 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:39.784970 2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 15:01:39.785529 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:39.785489 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-chsnd" Jul 07 15:01:40.098685 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:40.098402 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:40.098834 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:40.098749 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:40.373615 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:40.373553 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:01:40.374108 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:40.373925 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" event={"ID":"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e","Type":"ContainerStarted","Data":"9c6aa1aeebca4053e3e76c79cb5054229b3ded0b67dfd4909cfdb5dfccf59741"} Jul 07 15:01:40.374227 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:40.374213 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:40.374415 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:40.374393 2565 scope.go:117] "RemoveContainer" containerID="f61de84f7a6d1e602f1757e37d00b4e60bad2b2ad07551cdb92d716e21c3445c" Jul 07 15:01:40.375858 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:40.375832 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qz6xz" event={"ID":"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe","Type":"ContainerStarted","Data":"2f5003e1453772f89b2d5aa9b783495e4a74ad1799118f140f4eddca8a20f031"} Jul 07 15:01:40.389746 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:40.389728 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:41.101356 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:41.101323 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:41.101553 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:41.101430 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:41.101553 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:41.101479 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:41.101657 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:41.101561 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:41.381648 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:41.381584 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:01:41.382102 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:41.381953 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" event={"ID":"2def56c2-e1cd-4af7-8d6b-00e56dc39e1e","Type":"ContainerStarted","Data":"e455ab2b09be60044d9b7e621216b1bc77e5252989671f4d792098a94c291e62"} Jul 07 15:01:41.382178 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:41.382136 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:41.382278 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:41.382256 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:41.383962 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:41.383927 2565 generic.go:356] "Generic (PLEG): container finished" podID="15be5c3f-36e0-4a8d-be13-09a8cc5e73fe" containerID="2f5003e1453772f89b2d5aa9b783495e4a74ad1799118f140f4eddca8a20f031" exitCode=0 Jul 07 15:01:41.384045 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:41.383965 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qz6xz" event={"ID":"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe","Type":"ContainerDied","Data":"2f5003e1453772f89b2d5aa9b783495e4a74ad1799118f140f4eddca8a20f031"} Jul 07 15:01:41.397181 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:41.397153 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:01:41.450570 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:41.450518 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" podStartSLOduration=11.249624988 podStartE2EDuration="30.450500914s" totalImagesPullingTime="19.200875926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:41.416134948 +0000 UTC m=+30.841523336" watchObservedRunningTime="2026-07-07 15:01:41.450500914 +0000 UTC m=+30.875889301" Jul 07 15:01:42.098024 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:42.098004 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:42.098146 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:42.098127 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:42.388757 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:42.388673 2565 generic.go:356] "Generic (PLEG): container finished" podID="15be5c3f-36e0-4a8d-be13-09a8cc5e73fe" containerID="bba551cf0dbc9ef7349291e6f6172e6bc7b0b99663522d54c0f96e9200ced032" exitCode=0 Jul 07 15:01:42.389148 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:42.388751 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qz6xz" event={"ID":"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe","Type":"ContainerDied","Data":"bba551cf0dbc9ef7349291e6f6172e6bc7b0b99663522d54c0f96e9200ced032"} Jul 07 15:01:43.098827 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:43.098006 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:43.098827 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:43.098370 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:43.098827 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:43.098586 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:43.099140 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:43.099002 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:43.113042 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:43.113016 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-66m5k"] Jul 07 15:01:43.113893 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:43.113872 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j52v9"] Jul 07 15:01:43.114625 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:43.114602 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kfp7w"] Jul 07 15:01:43.114735 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:43.114688 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:43.114781 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:43.114765 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:43.390423 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:43.390397 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:43.390883 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:43.390396 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:43.390883 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:43.390497 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:43.390883 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:43.390583 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:44.395389 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:44.395293 2565 generic.go:356] "Generic (PLEG): container finished" podID="15be5c3f-36e0-4a8d-be13-09a8cc5e73fe" containerID="72a6246ed87828594055a932cf91de94038c79bb714d15db4375a955a7404b8e" exitCode=0 Jul 07 15:01:44.395389 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:44.395352 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qz6xz" event={"ID":"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe","Type":"ContainerDied","Data":"72a6246ed87828594055a932cf91de94038c79bb714d15db4375a955a7404b8e"} Jul 07 15:01:44.715644 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:44.715610 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs\") pod \"network-metrics-daemon-kfp7w\" (UID: \"3dece03e-eb3a-4661-855f-f37fca6d1923\") " pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:44.715809 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:44.715724 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:44.715809 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:44.715777 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs podName:3dece03e-eb3a-4661-855f-f37fca6d1923 nodeName:}" failed. No retries permitted until 2026-07-07 15:02:16.715763696 +0000 UTC m=+66.141152063 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs") pod "network-metrics-daemon-kfp7w" (UID: "3dece03e-eb3a-4661-855f-f37fca6d1923") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:44.816387 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:44.816346 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:44.816553 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:44.816473 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:44.816553 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:44.816537 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret podName:ae612123-dc47-43fd-b0aa-93d175975cd6 nodeName:}" failed. No retries permitted until 2026-07-07 15:02:16.816520589 +0000 UTC m=+66.241908958 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret") pod "global-pull-secret-syncer-66m5k" (UID: "ae612123-dc47-43fd-b0aa-93d175975cd6") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:44.916926 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:44.916892 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gkgs\" (UniqueName: \"kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs\") pod \"network-check-target-j52v9\" (UID: \"8b6865e3-214f-4260-87bd-66c295f1e763\") " pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:44.917089 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:44.917052 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:01:44.917089 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:44.917077 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:01:44.917089 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:44.917086 2565 projected.go:196] Error preparing data for projected volume kube-api-access-9gkgs for pod openshift-network-diagnostics/network-check-target-j52v9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:44.917213 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:44.917136 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs podName:8b6865e3-214f-4260-87bd-66c295f1e763 nodeName:}" failed. No retries permitted until 2026-07-07 15:02:16.917122865 +0000 UTC m=+66.342511230 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gkgs" (UniqueName: "kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs") pod "network-check-target-j52v9" (UID: "8b6865e3-214f-4260-87bd-66c295f1e763") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:45.098599 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:45.098514 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:45.098731 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:45.098522 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:45.098731 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:45.098611 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:45.098795 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:45.098754 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:45.098795 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:45.098778 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:45.098911 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:45.098891 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:47.100560 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:47.100530 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:47.100975 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:47.100638 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:47.100975 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:47.100649 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfp7w" podUID="3dece03e-eb3a-4661-855f-f37fca6d1923" Jul 07 15:01:47.100975 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:47.100712 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j52v9" podUID="8b6865e3-214f-4260-87bd-66c295f1e763" Jul 07 15:01:47.100975 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:47.100727 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:47.100975 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:47.100828 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66m5k" podUID="ae612123-dc47-43fd-b0aa-93d175975cd6" Jul 07 15:01:48.104375 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.104306 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-205.ec2.internal" event="NodeReady" Jul 07 15:01:48.104736 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.104429 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 15:01:48.161878 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.161820 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6ghkd"] Jul 07 15:01:48.181771 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.181734 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zhxj5"] Jul 07 15:01:48.181969 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.181866 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.184522 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.184303 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 15:01:48.184642 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.184310 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6qs2h\"" Jul 07 15:01:48.184688 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.184391 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 15:01:48.203609 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.203582 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6ghkd"] Jul 07 15:01:48.203743 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.203634 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zhxj5"] Jul 07 15:01:48.203743 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.203668 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dvpnf"] Jul 07 15:01:48.203850 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.203781 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.207617 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.207596 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 15:01:48.207989 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.207840 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 15:01:48.207989 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.207872 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4zds6\"" Jul 07 15:01:48.207989 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.207880 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 15:01:48.208181 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.208006 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 15:01:48.225568 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.225544 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dvpnf"] Jul 07 15:01:48.225685 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.225666 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dvpnf" Jul 07 15:01:48.228289 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.228269 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 15:01:48.228489 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.228294 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 15:01:48.228489 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.228425 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mt4rv\"" Jul 07 15:01:48.228489 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.228355 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 15:01:48.344238 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.344205 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/141b7ee7-5826-40f9-80ba-65febbba86ca-data-volume\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.344238 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.344241 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/141b7ee7-5826-40f9-80ba-65febbba86ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.344465 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.344272 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/141b7ee7-5826-40f9-80ba-65febbba86ca-crio-socket\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.344465 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.344344 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqfs4\" (UniqueName: \"kubernetes.io/projected/d0ca0927-1d4e-4dc0-a065-68a8d745cdc3-kube-api-access-sqfs4\") pod \"ingress-canary-dvpnf\" (UID: \"d0ca0927-1d4e-4dc0-a065-68a8d745cdc3\") " pod="openshift-ingress-canary/ingress-canary-dvpnf" Jul 07 15:01:48.344465 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.344392 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6139731-a6d0-45e4-b16a-a1743d67f6f1-tmp-dir\") pod \"dns-default-6ghkd\" (UID: \"e6139731-a6d0-45e4-b16a-a1743d67f6f1\") " pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.344465 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.344425 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e6139731-a6d0-45e4-b16a-a1743d67f6f1-config-volume\") pod \"dns-default-6ghkd\" (UID: \"e6139731-a6d0-45e4-b16a-a1743d67f6f1\") " pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.344465 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.344453 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl8fc\" (UniqueName: \"kubernetes.io/projected/e6139731-a6d0-45e4-b16a-a1743d67f6f1-kube-api-access-kl8fc\") pod \"dns-default-6ghkd\" (UID: \"e6139731-a6d0-45e4-b16a-a1743d67f6f1\") " pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.344677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.344564 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d0ca0927-1d4e-4dc0-a065-68a8d745cdc3-cert\") pod \"ingress-canary-dvpnf\" (UID: \"d0ca0927-1d4e-4dc0-a065-68a8d745cdc3\") " pod="openshift-ingress-canary/ingress-canary-dvpnf" Jul 07 15:01:48.344677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.344601 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/141b7ee7-5826-40f9-80ba-65febbba86ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.344677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.344638 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cchz9\" (UniqueName: \"kubernetes.io/projected/141b7ee7-5826-40f9-80ba-65febbba86ca-kube-api-access-cchz9\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.344677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.344669 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6139731-a6d0-45e4-b16a-a1743d67f6f1-metrics-tls\") pod \"dns-default-6ghkd\" (UID: \"e6139731-a6d0-45e4-b16a-a1743d67f6f1\") " pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.445579 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.445539 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/141b7ee7-5826-40f9-80ba-65febbba86ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.445769 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.445687 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/141b7ee7-5826-40f9-80ba-65febbba86ca-crio-socket\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.445769 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.445748 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqfs4\" (UniqueName: \"kubernetes.io/projected/d0ca0927-1d4e-4dc0-a065-68a8d745cdc3-kube-api-access-sqfs4\") pod \"ingress-canary-dvpnf\" (UID: \"d0ca0927-1d4e-4dc0-a065-68a8d745cdc3\") " pod="openshift-ingress-canary/ingress-canary-dvpnf" Jul 07 15:01:48.445902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.445787 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6139731-a6d0-45e4-b16a-a1743d67f6f1-tmp-dir\") pod \"dns-default-6ghkd\" (UID: \"e6139731-a6d0-45e4-b16a-a1743d67f6f1\") " pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.445902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.445813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e6139731-a6d0-45e4-b16a-a1743d67f6f1-config-volume\") pod \"dns-default-6ghkd\" (UID: \"e6139731-a6d0-45e4-b16a-a1743d67f6f1\") " pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.445902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.445837 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl8fc\" (UniqueName: \"kubernetes.io/projected/e6139731-a6d0-45e4-b16a-a1743d67f6f1-kube-api-access-kl8fc\") pod \"dns-default-6ghkd\" (UID: \"e6139731-a6d0-45e4-b16a-a1743d67f6f1\") " pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.445902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.445890 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d0ca0927-1d4e-4dc0-a065-68a8d745cdc3-cert\") pod \"ingress-canary-dvpnf\" (UID: \"d0ca0927-1d4e-4dc0-a065-68a8d745cdc3\") " pod="openshift-ingress-canary/ingress-canary-dvpnf" Jul 07 15:01:48.446131 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.445915 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/141b7ee7-5826-40f9-80ba-65febbba86ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.446131 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.445964 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cchz9\" (UniqueName: \"kubernetes.io/projected/141b7ee7-5826-40f9-80ba-65febbba86ca-kube-api-access-cchz9\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.446131 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.445986 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/141b7ee7-5826-40f9-80ba-65febbba86ca-crio-socket\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.446131 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.445991 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6139731-a6d0-45e4-b16a-a1743d67f6f1-metrics-tls\") pod \"dns-default-6ghkd\" (UID: \"e6139731-a6d0-45e4-b16a-a1743d67f6f1\") " pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.446329 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.446286 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/141b7ee7-5826-40f9-80ba-65febbba86ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.446381 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.446337 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6139731-a6d0-45e4-b16a-a1743d67f6f1-tmp-dir\") pod \"dns-default-6ghkd\" (UID: \"e6139731-a6d0-45e4-b16a-a1743d67f6f1\") " pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.446437 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.446404 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/141b7ee7-5826-40f9-80ba-65febbba86ca-data-volume\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.446865 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.446692 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/141b7ee7-5826-40f9-80ba-65febbba86ca-data-volume\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.446992 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.446979 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e6139731-a6d0-45e4-b16a-a1743d67f6f1-config-volume\") pod \"dns-default-6ghkd\" (UID: \"e6139731-a6d0-45e4-b16a-a1743d67f6f1\") " pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.451315 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.451289 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/141b7ee7-5826-40f9-80ba-65febbba86ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.451315 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.451304 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6139731-a6d0-45e4-b16a-a1743d67f6f1-metrics-tls\") pod \"dns-default-6ghkd\" (UID: \"e6139731-a6d0-45e4-b16a-a1743d67f6f1\") " pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.451529 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.451356 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d0ca0927-1d4e-4dc0-a065-68a8d745cdc3-cert\") pod \"ingress-canary-dvpnf\" (UID: \"d0ca0927-1d4e-4dc0-a065-68a8d745cdc3\") " pod="openshift-ingress-canary/ingress-canary-dvpnf" Jul 07 15:01:48.456593 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.456566 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqfs4\" (UniqueName: \"kubernetes.io/projected/d0ca0927-1d4e-4dc0-a065-68a8d745cdc3-kube-api-access-sqfs4\") pod \"ingress-canary-dvpnf\" (UID: \"d0ca0927-1d4e-4dc0-a065-68a8d745cdc3\") " pod="openshift-ingress-canary/ingress-canary-dvpnf" Jul 07 15:01:48.457041 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.457004 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cchz9\" (UniqueName: \"kubernetes.io/projected/141b7ee7-5826-40f9-80ba-65febbba86ca-kube-api-access-cchz9\") pod \"insights-runtime-extractor-zhxj5\" (UID: \"141b7ee7-5826-40f9-80ba-65febbba86ca\") " pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.457206 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.457181 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl8fc\" (UniqueName: \"kubernetes.io/projected/e6139731-a6d0-45e4-b16a-a1743d67f6f1-kube-api-access-kl8fc\") pod \"dns-default-6ghkd\" (UID: \"e6139731-a6d0-45e4-b16a-a1743d67f6f1\") " pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.494242 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.494207 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:48.513418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.513388 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zhxj5" Jul 07 15:01:48.535630 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:48.535603 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dvpnf" Jul 07 15:01:49.102054 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:49.102023 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:01:49.102245 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:49.102029 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:01:49.102245 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:49.102029 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:01:49.106452 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:49.106271 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 15:01:49.106452 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:49.106348 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 15:01:49.106452 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:49.106358 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 15:01:49.106452 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:49.106271 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 15:01:49.107236 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:49.106531 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bchjv\"" Jul 07 15:01:49.107393 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:49.107372 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ghr77\"" Jul 07 15:01:49.821490 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:49.821206 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dvpnf"] Jul 07 15:01:49.826207 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:49.826179 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ca0927_1d4e_4dc0_a065_68a8d745cdc3.slice/crio-6fae10eb0bbe9aab3f3a15d69d801c0bf45820a91c2d4686de87fcb1ee789f45 WatchSource:0}: Error finding container 6fae10eb0bbe9aab3f3a15d69d801c0bf45820a91c2d4686de87fcb1ee789f45: Status 404 returned error can't find the container with id 6fae10eb0bbe9aab3f3a15d69d801c0bf45820a91c2d4686de87fcb1ee789f45 Jul 07 15:01:49.829711 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:49.829643 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zhxj5"] Jul 07 15:01:49.834356 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:49.834335 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141b7ee7_5826_40f9_80ba_65febbba86ca.slice/crio-70f8d21fd9ec8bf33546a81807839a4900102e870d0d74004b12ac397855b56c WatchSource:0}: Error finding container 70f8d21fd9ec8bf33546a81807839a4900102e870d0d74004b12ac397855b56c: Status 404 returned error can't find the container with id 70f8d21fd9ec8bf33546a81807839a4900102e870d0d74004b12ac397855b56c Jul 07 15:01:49.836593 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:49.836518 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6ghkd"] Jul 07 15:01:49.840803 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:49.840770 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6139731_a6d0_45e4_b16a_a1743d67f6f1.slice/crio-9832ba4c690bb4e712d6757a65340a199d84bb546830722cd0c52c4da5081baf WatchSource:0}: Error finding container 9832ba4c690bb4e712d6757a65340a199d84bb546830722cd0c52c4da5081baf: Status 404 returned error can't find the container with id 9832ba4c690bb4e712d6757a65340a199d84bb546830722cd0c52c4da5081baf Jul 07 15:01:50.413516 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.413271 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6ghkd" event={"ID":"e6139731-a6d0-45e4-b16a-a1743d67f6f1","Type":"ContainerStarted","Data":"9832ba4c690bb4e712d6757a65340a199d84bb546830722cd0c52c4da5081baf"} Jul 07 15:01:50.416732 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.416700 2565 generic.go:356] "Generic (PLEG): container finished" podID="15be5c3f-36e0-4a8d-be13-09a8cc5e73fe" containerID="7e09f3af19ce6541525d208f5fca296137c60ea5642f1da24d8585f226b5b632" exitCode=0 Jul 07 15:01:50.416951 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.416796 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qz6xz" event={"ID":"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe","Type":"ContainerDied","Data":"7e09f3af19ce6541525d208f5fca296137c60ea5642f1da24d8585f226b5b632"} Jul 07 15:01:50.418694 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.418640 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dvpnf" event={"ID":"d0ca0927-1d4e-4dc0-a065-68a8d745cdc3","Type":"ContainerStarted","Data":"6fae10eb0bbe9aab3f3a15d69d801c0bf45820a91c2d4686de87fcb1ee789f45"} Jul 07 15:01:50.421188 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.421165 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zhxj5" event={"ID":"141b7ee7-5826-40f9-80ba-65febbba86ca","Type":"ContainerStarted","Data":"70f8d21fd9ec8bf33546a81807839a4900102e870d0d74004b12ac397855b56c"} Jul 07 15:01:50.463761 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.462859 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58755c6c75-4dgzc"] Jul 07 15:01:50.478901 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.478866 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.482851 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.481373 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58755c6c75-4dgzc"] Jul 07 15:01:50.483448 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.483428 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 15:01:50.483780 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.483761 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 15:01:50.484077 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.484064 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wt8t7\"" Jul 07 15:01:50.484310 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.484293 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 15:01:50.484746 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.484727 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 15:01:50.484955 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.484925 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 15:01:50.485558 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.485529 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 15:01:50.485660 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.485538 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 15:01:50.669201 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.669112 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-service-ca\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.669201 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.669170 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-oauth-serving-cert\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.669406 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.669242 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-oauth-config\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.669406 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.669277 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-serving-cert\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.669406 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.669302 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfkfs\" (UniqueName: \"kubernetes.io/projected/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-kube-api-access-dfkfs\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.669406 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.669333 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-config\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.770902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.770236 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-config\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.770902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.770304 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-service-ca\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.770902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.770336 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-oauth-serving-cert\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.770902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.770403 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-oauth-config\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.770902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.770433 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-serving-cert\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.770902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.770455 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfkfs\" (UniqueName: \"kubernetes.io/projected/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-kube-api-access-dfkfs\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.771498 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.771332 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-oauth-serving-cert\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.772393 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.772126 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-service-ca\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.772929 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.772889 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-config\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.776843 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.776801 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-oauth-config\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.776843 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.776836 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-serving-cert\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.786885 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.786854 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfkfs\" (UniqueName: \"kubernetes.io/projected/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-kube-api-access-dfkfs\") pod \"console-58755c6c75-4dgzc\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.796607 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.796584 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:01:50.994545 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:50.994514 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58755c6c75-4dgzc"] Jul 07 15:01:50.997885 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:50.997854 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6ee37f_e261_4fa0_9fa1_a035b1bcd63a.slice/crio-5834bc23f3d156c061936ec4aaf365e9aaac1061f3ef38e977e3f15b4abb60fb WatchSource:0}: Error finding container 5834bc23f3d156c061936ec4aaf365e9aaac1061f3ef38e977e3f15b4abb60fb: Status 404 returned error can't find the container with id 5834bc23f3d156c061936ec4aaf365e9aaac1061f3ef38e977e3f15b4abb60fb Jul 07 15:01:51.430919 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:51.429903 2565 generic.go:356] "Generic (PLEG): container finished" podID="15be5c3f-36e0-4a8d-be13-09a8cc5e73fe" containerID="2dd9cf9a12ba0dafe594aca19c03383d6c6a3d088907978571448f109955673d" exitCode=0 Jul 07 15:01:51.430919 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:51.430047 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qz6xz" event={"ID":"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe","Type":"ContainerDied","Data":"2dd9cf9a12ba0dafe594aca19c03383d6c6a3d088907978571448f109955673d"} Jul 07 15:01:51.432860 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:51.432795 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58755c6c75-4dgzc" event={"ID":"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a","Type":"ContainerStarted","Data":"5834bc23f3d156c061936ec4aaf365e9aaac1061f3ef38e977e3f15b4abb60fb"} Jul 07 15:01:54.574676 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:54.574500 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g"] Jul 07 15:01:54.708302 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:54.708265 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g"] Jul 07 15:01:54.708460 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:54.708374 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g" Jul 07 15:01:54.710916 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:54.710889 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 15:01:54.711047 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:54.711003 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-pqbs9\"" Jul 07 15:01:54.805260 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:54.805224 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/05477565-c157-422e-9691-b00ca6317b58-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-pnr2g\" (UID: \"05477565-c157-422e-9691-b00ca6317b58\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g" Jul 07 15:01:54.905923 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:54.905861 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/05477565-c157-422e-9691-b00ca6317b58-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-pnr2g\" (UID: \"05477565-c157-422e-9691-b00ca6317b58\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g" Jul 07 15:01:54.906035 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:54.905993 2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 07 15:01:54.906072 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:01:54.906049 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05477565-c157-422e-9691-b00ca6317b58-tls-certificates podName:05477565-c157-422e-9691-b00ca6317b58 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:55.406035493 +0000 UTC m=+44.831423859 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/05477565-c157-422e-9691-b00ca6317b58-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-pnr2g" (UID: "05477565-c157-422e-9691-b00ca6317b58") : secret "prometheus-operator-admission-webhook-tls" not found Jul 07 15:01:55.409730 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:55.409697 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/05477565-c157-422e-9691-b00ca6317b58-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-pnr2g\" (UID: \"05477565-c157-422e-9691-b00ca6317b58\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g" Jul 07 15:01:55.413047 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:55.413016 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/05477565-c157-422e-9691-b00ca6317b58-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-pnr2g\" (UID: \"05477565-c157-422e-9691-b00ca6317b58\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g" Jul 07 15:01:55.617296 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:55.617262 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g" Jul 07 15:01:57.820275 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:57.820246 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g"] Jul 07 15:01:57.824123 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:57.823754 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05477565_c157_422e_9691_b00ca6317b58.slice/crio-b65b121735d2cac333f6a626618cfa49f911a968e3327a39a00bf57448797577 WatchSource:0}: Error finding container b65b121735d2cac333f6a626618cfa49f911a968e3327a39a00bf57448797577: Status 404 returned error can't find the container with id b65b121735d2cac333f6a626618cfa49f911a968e3327a39a00bf57448797577 Jul 07 15:01:58.453714 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.453680 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dvpnf" event={"ID":"d0ca0927-1d4e-4dc0-a065-68a8d745cdc3","Type":"ContainerStarted","Data":"fea7124ab3d32aa148466bd9384c42eeb3e8c7c583e99c93bcee7250f6b60713"} Jul 07 15:01:58.455059 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.455026 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g" event={"ID":"05477565-c157-422e-9691-b00ca6317b58","Type":"ContainerStarted","Data":"b65b121735d2cac333f6a626618cfa49f911a968e3327a39a00bf57448797577"} Jul 07 15:01:58.457073 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.456983 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6ghkd" event={"ID":"e6139731-a6d0-45e4-b16a-a1743d67f6f1","Type":"ContainerStarted","Data":"b5bbea78720e750fcaea5838ca6951c487c1ef303a096f7e450278b4cc2a10e5"} Jul 07 15:01:58.457073 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.457033 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6ghkd" event={"ID":"e6139731-a6d0-45e4-b16a-a1743d67f6f1","Type":"ContainerStarted","Data":"94e33f9917dd930e23a8c8cdc397d41cc9175ebc5062bdd2a84bb998cb69b69c"} Jul 07 15:01:58.457263 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.457080 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6ghkd" Jul 07 15:01:58.460752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.460731 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qz6xz" event={"ID":"15be5c3f-36e0-4a8d-be13-09a8cc5e73fe","Type":"ContainerStarted","Data":"b32048b7992ac2e35028c8f21845559edafbed5b375c4f74d1302c4b6899d439"} Jul 07 15:01:58.462228 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.462206 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zhxj5" event={"ID":"141b7ee7-5826-40f9-80ba-65febbba86ca","Type":"ContainerStarted","Data":"58e8db746f0945cfacb674407d91e9a8ae63f6966146b94be7cfd686feb39f8b"} Jul 07 15:01:58.463683 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.463661 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58755c6c75-4dgzc" event={"ID":"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a","Type":"ContainerStarted","Data":"9d58162aeb47a6827f3c47f10f3d4c5ace79e1f7024cc6a71049c183e1316af2"} Jul 07 15:01:58.472874 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.472812 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dvpnf" podStartSLOduration=2.663170602 podStartE2EDuration="10.472799327s" totalImagesPullingTime="7.809628725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:58.471714231 +0000 UTC m=+47.897102619" watchObservedRunningTime="2026-07-07 15:01:58.472799327 +0000 UTC m=+47.898187714" Jul 07 15:01:58.490986 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.490917 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6ghkd" podStartSLOduration=3.063641654 podStartE2EDuration="10.490903622s" totalImagesPullingTime="7.427261968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:58.490804474 +0000 UTC m=+47.916192878" watchObservedRunningTime="2026-07-07 15:01:58.490903622 +0000 UTC m=+47.916292010" Jul 07 15:01:58.522465 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.522420 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qz6xz" podStartSLOduration=14.592731082 podStartE2EDuration="47.522402942s" totalImagesPullingTime="32.92967186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:01:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:58.521257304 +0000 UTC m=+47.946645692" watchObservedRunningTime="2026-07-07 15:01:58.522402942 +0000 UTC m=+47.947791329" Jul 07 15:01:58.544134 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.544088 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58755c6c75-4dgzc" podStartSLOduration=1.8634317839999999 podStartE2EDuration="8.544072506s" totalImagesPullingTime="6.680640722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:58.543670772 +0000 UTC m=+47.969059156" watchObservedRunningTime="2026-07-07 15:01:58.544072506 +0000 UTC m=+47.969460895" Jul 07 15:01:58.651565 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.651515 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5585cf8d45-h6p5f"] Jul 07 15:01:58.668927 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.668880 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5585cf8d45-h6p5f"] Jul 07 15:01:58.669105 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.669007 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.679821 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.679797 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 15:01:58.737781 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.737748 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-oauth-serving-cert\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.737928 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.737811 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ckpz\" (UniqueName: \"kubernetes.io/projected/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-kube-api-access-9ckpz\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.737928 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.737889 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-serving-cert\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.737928 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.737909 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-oauth-config\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.737928 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.737924 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-service-ca\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.738148 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.737997 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-config\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.738148 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.738033 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-trusted-ca-bundle\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.838520 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.838493 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-oauth-serving-cert\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.838888 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.838566 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ckpz\" (UniqueName: \"kubernetes.io/projected/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-kube-api-access-9ckpz\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.838888 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.838612 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-serving-cert\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.838888 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.838647 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-oauth-config\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.838888 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.838671 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-service-ca\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.838888 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.838702 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-config\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.838888 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.838727 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-trusted-ca-bundle\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.839541 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.839513 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-config\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.839624 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.839513 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-service-ca\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.839624 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.839590 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-oauth-serving-cert\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.840171 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.840150 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-trusted-ca-bundle\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.840968 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.840932 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-oauth-config\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.841210 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.841191 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-serving-cert\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.847518 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.847493 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ckpz\" (UniqueName: \"kubernetes.io/projected/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-kube-api-access-9ckpz\") pod \"console-5585cf8d45-h6p5f\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:58.982174 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:58.982089 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:01:59.122996 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:59.122967 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5585cf8d45-h6p5f"] Jul 07 15:01:59.344155 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:01:59.344088 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod097f8a79_ba3b_493e_8c0e_c1ae6444ca07.slice/crio-f7d156314f497ab221d6fb6e7e57d9675ff36caa6ff12c06ca7c6ede1d4111f8 WatchSource:0}: Error finding container f7d156314f497ab221d6fb6e7e57d9675ff36caa6ff12c06ca7c6ede1d4111f8: Status 404 returned error can't find the container with id f7d156314f497ab221d6fb6e7e57d9675ff36caa6ff12c06ca7c6ede1d4111f8 Jul 07 15:01:59.469284 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:59.469242 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5585cf8d45-h6p5f" event={"ID":"097f8a79-ba3b-493e-8c0e-c1ae6444ca07","Type":"ContainerStarted","Data":"20b26e68d816765383f3e17eeba777b888ddc3288ba3534eee41db54e0d6237c"} Jul 07 15:01:59.469443 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:59.469309 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5585cf8d45-h6p5f" event={"ID":"097f8a79-ba3b-493e-8c0e-c1ae6444ca07","Type":"ContainerStarted","Data":"f7d156314f497ab221d6fb6e7e57d9675ff36caa6ff12c06ca7c6ede1d4111f8"} Jul 07 15:01:59.471401 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:59.471368 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zhxj5" event={"ID":"141b7ee7-5826-40f9-80ba-65febbba86ca","Type":"ContainerStarted","Data":"d6e17d674ca9f91112174cc429455b1deb885a468ea727bbb287f79761ab901b"} Jul 07 15:01:59.505484 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:01:59.505431 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5585cf8d45-h6p5f" podStartSLOduration=1.5054153750000001 podStartE2EDuration="1.505415375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:59.505330571 +0000 UTC m=+48.930718960" watchObservedRunningTime="2026-07-07 15:01:59.505415375 +0000 UTC m=+48.930803762" Jul 07 15:02:00.476482 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:00.476436 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g" event={"ID":"05477565-c157-422e-9691-b00ca6317b58","Type":"ContainerStarted","Data":"ee53b39fe6304d6f2bb3ffabc63a64fd123589955fc9756690479ecef1d95087"} Jul 07 15:02:00.496210 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:00.496142 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g" podStartSLOduration=4.616783116 podStartE2EDuration="6.496121145s" totalImagesPullingTime="1.879338029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:00.495255218 +0000 UTC m=+49.920643620" watchObservedRunningTime="2026-07-07 15:02:00.496121145 +0000 UTC m=+49.921509536" Jul 07 15:02:00.797537 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:00.797439 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:02:00.797537 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:00.797538 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:02:00.798955 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:00.798914 2565 patch_prober.go:28] interesting pod/console-58755c6c75-4dgzc container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body= Jul 07 15:02:00.799055 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:00.798981 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-58755c6c75-4dgzc" podUID="cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" Jul 07 15:02:01.479781 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:01.479744 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g" Jul 07 15:02:01.485064 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:01.485033 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pnr2g" Jul 07 15:02:02.484431 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:02.484399 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zhxj5" event={"ID":"141b7ee7-5826-40f9-80ba-65febbba86ca","Type":"ContainerStarted","Data":"b10ebbb77cdc5b159ff60b909ae2d949e8c9ccc2c39470b6784cc0501892aeb6"} Jul 07 15:02:07.172839 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.172776 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zhxj5" podStartSLOduration=7.26463245 podStartE2EDuration="19.172760902s" totalImagesPullingTime="11.908128452s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:01:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:02.508082974 +0000 UTC m=+51.933471362" watchObservedRunningTime="2026-07-07 15:02:07.172760902 +0000 UTC m=+56.598149290" Jul 07 15:02:07.173575 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.173558 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5"] Jul 07 15:02:07.176220 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.176204 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.182643 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.182617 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 15:02:07.182643 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.182639 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 15:02:07.182843 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.182717 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 15:02:07.182843 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.182718 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 15:02:07.182843 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.182721 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 15:02:07.182843 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.182834 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bb9jz\"" Jul 07 15:02:07.183836 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.183798 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9"] Jul 07 15:02:07.185967 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.185932 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.189332 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.189309 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 15:02:07.189448 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.189369 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 15:02:07.189565 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.189530 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-wn8mb\"" Jul 07 15:02:07.189774 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.189756 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 15:02:07.191416 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.191398 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5"] Jul 07 15:02:07.201035 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.201017 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ctxm9"] Jul 07 15:02:07.202257 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.202233 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wncb9\" (UniqueName: \"kubernetes.io/projected/00f1ebed-3e42-4141-9c94-1b312a7c9c43-kube-api-access-wncb9\") pod \"openshift-state-metrics-54cf96dfcf-tdcz5\" (UID: \"00f1ebed-3e42-4141-9c94-1b312a7c9c43\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.202338 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.202291 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00f1ebed-3e42-4141-9c94-1b312a7c9c43-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-tdcz5\" (UID: \"00f1ebed-3e42-4141-9c94-1b312a7c9c43\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.202377 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.202339 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/00f1ebed-3e42-4141-9c94-1b312a7c9c43-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-tdcz5\" (UID: \"00f1ebed-3e42-4141-9c94-1b312a7c9c43\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.202419 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.202405 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00f1ebed-3e42-4141-9c94-1b312a7c9c43-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-tdcz5\" (UID: \"00f1ebed-3e42-4141-9c94-1b312a7c9c43\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.203136 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.203123 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.204605 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.204588 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9"] Jul 07 15:02:07.205972 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.205930 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 15:02:07.206075 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.205986 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 15:02:07.206075 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.206043 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5lxkv\"" Jul 07 15:02:07.206284 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.206272 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 15:02:07.303312 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303275 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.303312 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303313 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-textfile\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.303533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303337 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6c145f55-6798-421c-9d01-e6e42fe2a7f6-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.303533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303357 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzdgs\" (UniqueName: \"kubernetes.io/projected/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-kube-api-access-rzdgs\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.303533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303372 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c145f55-6798-421c-9d01-e6e42fe2a7f6-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.303533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303387 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdzs6\" (UniqueName: \"kubernetes.io/projected/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-api-access-rdzs6\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.303533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303405 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wncb9\" (UniqueName: \"kubernetes.io/projected/00f1ebed-3e42-4141-9c94-1b312a7c9c43-kube-api-access-wncb9\") pod \"openshift-state-metrics-54cf96dfcf-tdcz5\" (UID: \"00f1ebed-3e42-4141-9c94-1b312a7c9c43\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.303533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303421 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.303533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303439 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-metrics-client-ca\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.303533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303506 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.303795 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303557 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-root\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.303795 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303583 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.303795 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303628 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-tls\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.303795 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303674 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-accelerators-collector-config\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.303795 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303733 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00f1ebed-3e42-4141-9c94-1b312a7c9c43-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-tdcz5\" (UID: \"00f1ebed-3e42-4141-9c94-1b312a7c9c43\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.303795 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303794 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-sys\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.304020 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303814 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/00f1ebed-3e42-4141-9c94-1b312a7c9c43-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-tdcz5\" (UID: \"00f1ebed-3e42-4141-9c94-1b312a7c9c43\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.304020 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303836 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-wtmp\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.304020 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.303862 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00f1ebed-3e42-4141-9c94-1b312a7c9c43-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-tdcz5\" (UID: \"00f1ebed-3e42-4141-9c94-1b312a7c9c43\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.304020 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:02:07.303971 2565 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 07 15:02:07.304144 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:02:07.304023 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00f1ebed-3e42-4141-9c94-1b312a7c9c43-openshift-state-metrics-tls podName:00f1ebed-3e42-4141-9c94-1b312a7c9c43 nodeName:}" failed. No retries permitted until 2026-07-07 15:02:07.804008974 +0000 UTC m=+57.229397339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/00f1ebed-3e42-4141-9c94-1b312a7c9c43-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-tdcz5" (UID: "00f1ebed-3e42-4141-9c94-1b312a7c9c43") : secret "openshift-state-metrics-tls" not found Jul 07 15:02:07.304534 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.304518 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00f1ebed-3e42-4141-9c94-1b312a7c9c43-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-tdcz5\" (UID: \"00f1ebed-3e42-4141-9c94-1b312a7c9c43\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.306085 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.306070 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00f1ebed-3e42-4141-9c94-1b312a7c9c43-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-tdcz5\" (UID: \"00f1ebed-3e42-4141-9c94-1b312a7c9c43\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.316872 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.316852 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wncb9\" (UniqueName: \"kubernetes.io/projected/00f1ebed-3e42-4141-9c94-1b312a7c9c43-kube-api-access-wncb9\") pod \"openshift-state-metrics-54cf96dfcf-tdcz5\" (UID: \"00f1ebed-3e42-4141-9c94-1b312a7c9c43\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.404768 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.404734 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-wtmp\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.405000 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.404809 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.405000 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.404833 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-textfile\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.405000 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.404864 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6c145f55-6798-421c-9d01-e6e42fe2a7f6-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.405000 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.404892 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzdgs\" (UniqueName: \"kubernetes.io/projected/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-kube-api-access-rzdgs\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.405000 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.404903 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-wtmp\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.405000 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.404976 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c145f55-6798-421c-9d01-e6e42fe2a7f6-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.405306 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405007 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdzs6\" (UniqueName: \"kubernetes.io/projected/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-api-access-rdzs6\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.405306 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405035 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.405306 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405054 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-metrics-client-ca\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.405306 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405090 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.405306 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:02:07.405118 2565 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 07 15:02:07.405306 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405135 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-root\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.405306 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405171 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.405306 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405173 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-textfile\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.405306 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:02:07.405201 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-state-metrics-tls podName:6c145f55-6798-421c-9d01-e6e42fe2a7f6 nodeName:}" failed. No retries permitted until 2026-07-07 15:02:07.90518012 +0000 UTC m=+57.330568492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-pcrj9" (UID: "6c145f55-6798-421c-9d01-e6e42fe2a7f6") : secret "kube-state-metrics-tls" not found Jul 07 15:02:07.405306 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405246 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-root\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.405306 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405301 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-tls\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.405306 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405311 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6c145f55-6798-421c-9d01-e6e42fe2a7f6-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.405879 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405329 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-accelerators-collector-config\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.405879 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405361 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-sys\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.405879 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:02:07.405393 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 15:02:07.405879 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:02:07.405449 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-tls podName:c31fc1aa-2d1b-4e21-b2fc-603ce581ce50 nodeName:}" failed. No retries permitted until 2026-07-07 15:02:07.905432946 +0000 UTC m=+57.330821327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-tls") pod "node-exporter-ctxm9" (UID: "c31fc1aa-2d1b-4e21-b2fc-603ce581ce50") : secret "node-exporter-tls" not found Jul 07 15:02:07.405879 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405448 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-sys\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.405879 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405872 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-metrics-client-ca\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.406198 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405899 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-accelerators-collector-config\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.406198 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405931 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.406198 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.405999 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c145f55-6798-421c-9d01-e6e42fe2a7f6-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.407554 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.407535 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.407650 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.407578 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.423342 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.423279 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzdgs\" (UniqueName: \"kubernetes.io/projected/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-kube-api-access-rzdgs\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.435045 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.435023 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdzs6\" (UniqueName: \"kubernetes.io/projected/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-api-access-rdzs6\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.809911 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.809821 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/00f1ebed-3e42-4141-9c94-1b312a7c9c43-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-tdcz5\" (UID: \"00f1ebed-3e42-4141-9c94-1b312a7c9c43\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.812172 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.812142 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/00f1ebed-3e42-4141-9c94-1b312a7c9c43-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-tdcz5\" (UID: \"00f1ebed-3e42-4141-9c94-1b312a7c9c43\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:07.911016 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.910977 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:07.911176 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.911149 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-tls\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.913265 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.913237 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c31fc1aa-2d1b-4e21-b2fc-603ce581ce50-node-exporter-tls\") pod \"node-exporter-ctxm9\" (UID: \"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50\") " pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:07.913383 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:07.913364 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c145f55-6798-421c-9d01-e6e42fe2a7f6-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-pcrj9\" (UID: \"6c145f55-6798-421c-9d01-e6e42fe2a7f6\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:08.084839 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.084732 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" Jul 07 15:02:08.095724 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.095698 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" Jul 07 15:02:08.111906 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.111877 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ctxm9" Jul 07 15:02:08.240458 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.240436 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5"] Jul 07 15:02:08.242611 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:02:08.242586 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00f1ebed_3e42_4141_9c94_1b312a7c9c43.slice/crio-5f423bf6507de39babbcfee19f154353cf22a1ee6f170bf48f3fcf03e767aa52 WatchSource:0}: Error finding container 5f423bf6507de39babbcfee19f154353cf22a1ee6f170bf48f3fcf03e767aa52: Status 404 returned error can't find the container with id 5f423bf6507de39babbcfee19f154353cf22a1ee6f170bf48f3fcf03e767aa52 Jul 07 15:02:08.251800 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.251771 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9"] Jul 07 15:02:08.255569 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:02:08.255544 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c145f55_6798_421c_9d01_e6e42fe2a7f6.slice/crio-abd5c94397f77fbe03a7af50da7baa07734bfc9fc3f3316da37c5b9f06f72c03 WatchSource:0}: Error finding container abd5c94397f77fbe03a7af50da7baa07734bfc9fc3f3316da37c5b9f06f72c03: Status 404 returned error can't find the container with id abd5c94397f77fbe03a7af50da7baa07734bfc9fc3f3316da37c5b9f06f72c03 Jul 07 15:02:08.473358 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.473323 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6ghkd" Jul 07 15:02:08.507136 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.507102 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ctxm9" event={"ID":"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50","Type":"ContainerStarted","Data":"9665a8c6d551d8a0d37e21b19b2448cb88ec530b492c076db85f6c770bea3ce4"} Jul 07 15:02:08.508894 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.508868 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" event={"ID":"00f1ebed-3e42-4141-9c94-1b312a7c9c43","Type":"ContainerStarted","Data":"ae69550fdd26b5d84c7b97b597ca5ef2a364bd96b4634da2078acd20c6ee7b88"} Jul 07 15:02:08.509038 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.508916 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" event={"ID":"00f1ebed-3e42-4141-9c94-1b312a7c9c43","Type":"ContainerStarted","Data":"9ec83ee48071f308307ad020159377d7152d88e49fbe6abdb007beaf57459297"} Jul 07 15:02:08.509038 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.508958 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" event={"ID":"00f1ebed-3e42-4141-9c94-1b312a7c9c43","Type":"ContainerStarted","Data":"5f423bf6507de39babbcfee19f154353cf22a1ee6f170bf48f3fcf03e767aa52"} Jul 07 15:02:08.509920 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.509891 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" event={"ID":"6c145f55-6798-421c-9d01-e6e42fe2a7f6","Type":"ContainerStarted","Data":"abd5c94397f77fbe03a7af50da7baa07734bfc9fc3f3316da37c5b9f06f72c03"} Jul 07 15:02:08.982258 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.982222 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:02:08.982446 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.982283 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:02:08.987817 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:08.987793 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:02:09.520430 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.520218 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:02:09.527460 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.527432 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-67b5f99c8c-k666l"] Jul 07 15:02:09.530452 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.530429 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.538511 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.538239 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 15:02:09.538511 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.538242 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 15:02:09.538511 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.538348 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 15:02:09.540024 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.539171 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 15:02:09.540024 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.539473 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-a5j3n36dotaj6\"" Jul 07 15:02:09.542657 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.541741 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-67b5f99c8c-k666l"] Jul 07 15:02:09.545522 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.545480 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-jk2jc\"" Jul 07 15:02:09.545758 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.545740 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 15:02:09.625934 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.625899 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-tls\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.626190 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.625958 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.626190 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.626074 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.626190 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.626102 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.626190 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.626128 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-grpc-tls\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.626426 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.626220 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e12f9e4-29a4-4381-b884-db2d849aed43-metrics-client-ca\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.626426 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.626289 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbxmr\" (UniqueName: \"kubernetes.io/projected/3e12f9e4-29a4-4381-b884-db2d849aed43-kube-api-access-qbxmr\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.626426 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.626357 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.639845 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.639810 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58755c6c75-4dgzc"] Jul 07 15:02:09.727279 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.727243 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbxmr\" (UniqueName: \"kubernetes.io/projected/3e12f9e4-29a4-4381-b884-db2d849aed43-kube-api-access-qbxmr\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.727442 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.727310 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.727442 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.727360 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-tls\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.727442 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.727378 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.727442 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.727410 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.727442 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.727433 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.727706 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.727458 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-grpc-tls\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.727706 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.727505 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e12f9e4-29a4-4381-b884-db2d849aed43-metrics-client-ca\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.728324 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.728297 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e12f9e4-29a4-4381-b884-db2d849aed43-metrics-client-ca\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.730469 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.730421 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.730595 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.730467 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.730595 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.730568 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-tls\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.730789 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.730768 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-grpc-tls\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.730843 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.730796 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.731293 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.731273 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3e12f9e4-29a4-4381-b884-db2d849aed43-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.742482 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.742420 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbxmr\" (UniqueName: \"kubernetes.io/projected/3e12f9e4-29a4-4381-b884-db2d849aed43-kube-api-access-qbxmr\") pod \"thanos-querier-67b5f99c8c-k666l\" (UID: \"3e12f9e4-29a4-4381-b884-db2d849aed43\") " pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:09.848195 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:09.848106 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:10.271859 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:10.271825 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-67b5f99c8c-k666l"] Jul 07 15:02:10.519343 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:10.519307 2565 generic.go:356] "Generic (PLEG): container finished" podID="c31fc1aa-2d1b-4e21-b2fc-603ce581ce50" containerID="46135f64c43135d9ed33ef621f478b1b8bc0a579004758056e19c605250ec62c" exitCode=0 Jul 07 15:02:10.519494 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:10.519395 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ctxm9" event={"ID":"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50","Type":"ContainerDied","Data":"46135f64c43135d9ed33ef621f478b1b8bc0a579004758056e19c605250ec62c"} Jul 07 15:02:10.521618 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:10.521595 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" event={"ID":"00f1ebed-3e42-4141-9c94-1b312a7c9c43","Type":"ContainerStarted","Data":"ec6863e7faa4d28235bd6b2992540894521d4a516cf3e616c95144c1ec6170df"} Jul 07 15:02:10.523703 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:10.523681 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" event={"ID":"6c145f55-6798-421c-9d01-e6e42fe2a7f6","Type":"ContainerStarted","Data":"21e3bd7bc6e814b6071841f3222e47699e54a1973923dd7f7376445e3b0c33bc"} Jul 07 15:02:10.523787 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:10.523725 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" event={"ID":"6c145f55-6798-421c-9d01-e6e42fe2a7f6","Type":"ContainerStarted","Data":"28d4eb6fc8365e83e09b7bb0ac21897e52cd57549c2d0173eedc6064c39cf6a4"} Jul 07 15:02:10.523787 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:10.523753 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" event={"ID":"6c145f55-6798-421c-9d01-e6e42fe2a7f6","Type":"ContainerStarted","Data":"e2322949e0ffb55e728bfbe143a023bd16dad0ac3e7b772f6eca7b04940c9040"} Jul 07 15:02:10.524803 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:10.524780 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" event={"ID":"3e12f9e4-29a4-4381-b884-db2d849aed43","Type":"ContainerStarted","Data":"72281d3b53b29b422afc5a9303b1f066098013bc0276ddfe57f546188693fae0"} Jul 07 15:02:10.582488 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:10.582440 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-pcrj9" podStartSLOduration=1.727685015 podStartE2EDuration="3.582420116s" totalImagesPullingTime="1.854735101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:10.582286528 +0000 UTC m=+60.007674916" watchObservedRunningTime="2026-07-07 15:02:10.582420116 +0000 UTC m=+60.007808504" Jul 07 15:02:10.605396 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:10.605270 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tdcz5" podStartSLOduration=1.8661741429999998 podStartE2EDuration="3.605250753s" totalImagesPullingTime="1.73907661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:10.604471096 +0000 UTC m=+60.029859484" watchObservedRunningTime="2026-07-07 15:02:10.605250753 +0000 UTC m=+60.030639141" Jul 07 15:02:11.535365 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.535320 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ctxm9" event={"ID":"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50","Type":"ContainerStarted","Data":"c722a45b0b405524c643dbc605d9e28a285efb8ad4d39014be88e673a6bab49d"} Jul 07 15:02:11.535831 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.535414 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ctxm9" event={"ID":"c31fc1aa-2d1b-4e21-b2fc-603ce581ce50","Type":"ContainerStarted","Data":"90424e8d0b9ded033f733c7a6eee3ee19e5fd96c840b3bebe4ad250fb9b77409"} Jul 07 15:02:11.560019 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.559930 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ctxm9" podStartSLOduration=2.473398935 podStartE2EDuration="4.559915372s" totalImagesPullingTime="1.988464899s" totalInitContainerRuntime="98.051538ms" isStatefulPod=true podCreationTimestamp="2026-07-07 15:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:11.559209034 +0000 UTC m=+60.984597422" watchObservedRunningTime="2026-07-07 15:02:11.559915372 +0000 UTC m=+60.985303759" Jul 07 15:02:11.575781 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.575750 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7bfd95bbff-dq4f5"] Jul 07 15:02:11.578889 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.578858 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.582008 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.581933 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 15:02:11.582008 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.582000 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 15:02:11.582199 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.582024 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ce9mg0nbost9n\"" Jul 07 15:02:11.582259 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.582209 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 15:02:11.582259 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.582238 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 15:02:11.582359 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.582265 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-x5tmr\"" Jul 07 15:02:11.593582 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.593560 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7bfd95bbff-dq4f5"] Jul 07 15:02:11.644237 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.644201 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-secret-metrics-server-tls\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.644394 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.644353 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-audit-log\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.644668 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.644633 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-secret-metrics-server-client-certs\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.644738 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.644693 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-client-ca-bundle\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.644738 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.644720 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqlmq\" (UniqueName: \"kubernetes.io/projected/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-kube-api-access-gqlmq\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.644831 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.644748 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.644831 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.644773 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-metrics-server-audit-profiles\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.745474 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.745427 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-secret-metrics-server-client-certs\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.745474 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.745473 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-client-ca-bundle\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.745715 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.745491 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqlmq\" (UniqueName: \"kubernetes.io/projected/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-kube-api-access-gqlmq\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.745715 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.745519 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.745715 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.745552 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-metrics-server-audit-profiles\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.745854 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.745713 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-secret-metrics-server-tls\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.745854 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.745771 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-audit-log\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.746165 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.746139 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-audit-log\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.746485 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.746460 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.746731 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.746711 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-metrics-server-audit-profiles\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.748062 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.748039 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-secret-metrics-server-tls\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.748187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.748170 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-secret-metrics-server-client-certs\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.748228 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.748196 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-client-ca-bundle\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.754628 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.754605 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqlmq\" (UniqueName: \"kubernetes.io/projected/8ef5f4ac-fcc0-4174-bf67-2b29456a4988-kube-api-access-gqlmq\") pod \"metrics-server-7bfd95bbff-dq4f5\" (UID: \"8ef5f4ac-fcc0-4174-bf67-2b29456a4988\") " pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:11.890037 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:11.889889 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:12.038734 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:12.038705 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7bfd95bbff-dq4f5"] Jul 07 15:02:12.520742 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:02:12.520713 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ef5f4ac_fcc0_4174_bf67_2b29456a4988.slice/crio-2b352af5e80c4fbe1de5aca8f5fc957a013d1224eeade6fa97868026d4e55838 WatchSource:0}: Error finding container 2b352af5e80c4fbe1de5aca8f5fc957a013d1224eeade6fa97868026d4e55838: Status 404 returned error can't find the container with id 2b352af5e80c4fbe1de5aca8f5fc957a013d1224eeade6fa97868026d4e55838 Jul 07 15:02:12.539270 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:12.539242 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" event={"ID":"8ef5f4ac-fcc0-4174-bf67-2b29456a4988","Type":"ContainerStarted","Data":"2b352af5e80c4fbe1de5aca8f5fc957a013d1224eeade6fa97868026d4e55838"} Jul 07 15:02:12.864770 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:12.864688 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-95569c898-4w6xm"] Jul 07 15:02:12.883214 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:12.883191 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-95569c898-4w6xm"] Jul 07 15:02:12.883325 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:12.883276 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:12.958122 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:12.958089 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-service-ca\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:12.958247 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:12.958163 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-config\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:12.958282 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:12.958240 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-serving-cert\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:12.958333 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:12.958278 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-trusted-ca-bundle\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:12.958333 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:12.958299 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-oauth-serving-cert\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:12.958333 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:12.958324 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-847r4\" (UniqueName: \"kubernetes.io/projected/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-kube-api-access-847r4\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:12.958435 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:12.958367 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-oauth-config\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.058690 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.058667 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-config\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.058810 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.058717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-serving-cert\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.058810 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.058741 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-trusted-ca-bundle\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.058810 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.058765 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-oauth-serving-cert\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.058810 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.058792 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-847r4\" (UniqueName: \"kubernetes.io/projected/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-kube-api-access-847r4\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.059013 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.058843 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-oauth-config\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.059013 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.058882 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-service-ca\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.059480 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.059458 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-config\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.059583 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.059540 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-service-ca\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.059757 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.059735 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-trusted-ca-bundle\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.059798 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.059782 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-oauth-serving-cert\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.061192 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.061172 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-serving-cert\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.061267 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.061190 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-oauth-config\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.070904 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.070882 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-847r4\" (UniqueName: \"kubernetes.io/projected/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-kube-api-access-847r4\") pod \"console-95569c898-4w6xm\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.192863 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.192838 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:13.385624 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.385598 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-95569c898-4w6xm"] Jul 07 15:02:13.387236 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:02:13.387213 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf0efcb_6b99_4e92_ae58_bdf839edde2b.slice/crio-6a585e1c65221550cd065002fa8ab410fdc91f20da860245bb43cc3894da91eb WatchSource:0}: Error finding container 6a585e1c65221550cd065002fa8ab410fdc91f20da860245bb43cc3894da91eb: Status 404 returned error can't find the container with id 6a585e1c65221550cd065002fa8ab410fdc91f20da860245bb43cc3894da91eb Jul 07 15:02:13.408078 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.408055 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-snw9m" Jul 07 15:02:13.546079 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.546039 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" event={"ID":"3e12f9e4-29a4-4381-b884-db2d849aed43","Type":"ContainerStarted","Data":"34f8a7a7e70edfb0261ae57360f2adf1548971e2b7dd15eab14ff6b7b99ce0e2"} Jul 07 15:02:13.546494 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.546157 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" event={"ID":"3e12f9e4-29a4-4381-b884-db2d849aed43","Type":"ContainerStarted","Data":"e5849234823cc277114fcf5af1597fcb88047145fe05762d4730c0add1def341"} Jul 07 15:02:13.546494 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.546187 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" event={"ID":"3e12f9e4-29a4-4381-b884-db2d849aed43","Type":"ContainerStarted","Data":"a4d1a9415f93eb352dff2718208ff8a1e7652f835dd065d6b0ea5972fbfc0c20"} Jul 07 15:02:13.547677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.547647 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-95569c898-4w6xm" event={"ID":"8bf0efcb-6b99-4e92-ae58-bdf839edde2b","Type":"ContainerStarted","Data":"7c9a827a90004e0d2793b94f92a70a4ae1be0dda5ab6dd1a0438583b0e314c1c"} Jul 07 15:02:13.547780 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.547690 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-95569c898-4w6xm" event={"ID":"8bf0efcb-6b99-4e92-ae58-bdf839edde2b","Type":"ContainerStarted","Data":"6a585e1c65221550cd065002fa8ab410fdc91f20da860245bb43cc3894da91eb"} Jul 07 15:02:13.597474 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:13.597424 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-95569c898-4w6xm" podStartSLOduration=1.597408221 podStartE2EDuration="1.597408221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:02:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:13.597215687 +0000 UTC m=+63.022604089" watchObservedRunningTime="2026-07-07 15:02:13.597408221 +0000 UTC m=+63.022796600" Jul 07 15:02:14.556777 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:14.556686 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" event={"ID":"3e12f9e4-29a4-4381-b884-db2d849aed43","Type":"ContainerStarted","Data":"1c37d1c610a13f36c0f279637395e1446f7d07aab145c03a1526ba35ce520fcd"} Jul 07 15:02:14.556777 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:14.556753 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" event={"ID":"3e12f9e4-29a4-4381-b884-db2d849aed43","Type":"ContainerStarted","Data":"5a24e3271ab4ebe776ab0f58006cff2c81e5ab532cf10d24d228ae59b12666b8"} Jul 07 15:02:14.557325 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:14.556782 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" event={"ID":"3e12f9e4-29a4-4381-b884-db2d849aed43","Type":"ContainerStarted","Data":"7c3296df506e3388a7337342cfffad32ce9315942298a0e9282b09578a767a9c"} Jul 07 15:02:14.587850 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:14.587777 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" podStartSLOduration=1.949536801 podStartE2EDuration="5.58775684s" totalImagesPullingTime="3.638220039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:02:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:14.585875402 +0000 UTC m=+64.011263794" watchObservedRunningTime="2026-07-07 15:02:14.58775684 +0000 UTC m=+64.013145231" Jul 07 15:02:15.561851 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:15.561815 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:16.566123 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:16.566086 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" event={"ID":"8ef5f4ac-fcc0-4174-bf67-2b29456a4988","Type":"ContainerStarted","Data":"225c84eec965c45cb1faafedbc268b1c75079d3b1cb2aec743d2e576299d4010"} Jul 07 15:02:16.585265 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:16.585217 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" podStartSLOduration=2.467663285 podStartE2EDuration="5.585199734s" totalImagesPullingTime="3.117536449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:16.583580933 +0000 UTC m=+66.008969382" watchObservedRunningTime="2026-07-07 15:02:16.585199734 +0000 UTC m=+66.010588127" Jul 07 15:02:16.794386 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:16.794352 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs\") pod \"network-metrics-daemon-kfp7w\" (UID: \"3dece03e-eb3a-4661-855f-f37fca6d1923\") " pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:02:16.797271 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:16.797250 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 15:02:16.807017 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:16.806993 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dece03e-eb3a-4661-855f-f37fca6d1923-metrics-certs\") pod \"network-metrics-daemon-kfp7w\" (UID: \"3dece03e-eb3a-4661-855f-f37fca6d1923\") " pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:02:16.895409 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:16.895316 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:02:16.898164 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:16.898147 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 15:02:16.908492 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:16.908476 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae612123-dc47-43fd-b0aa-93d175975cd6-original-pull-secret\") pod \"global-pull-secret-syncer-66m5k\" (UID: \"ae612123-dc47-43fd-b0aa-93d175975cd6\") " pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:02:16.996103 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:16.996069 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gkgs\" (UniqueName: \"kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs\") pod \"network-check-target-j52v9\" (UID: \"8b6865e3-214f-4260-87bd-66c295f1e763\") " pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:02:16.999126 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:16.999107 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 15:02:17.009375 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:17.009353 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 15:02:17.017070 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:17.017053 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ghr77\"" Jul 07 15:02:17.019218 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:17.019199 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gkgs\" (UniqueName: \"kubernetes.io/projected/8b6865e3-214f-4260-87bd-66c295f1e763-kube-api-access-9gkgs\") pod \"network-check-target-j52v9\" (UID: \"8b6865e3-214f-4260-87bd-66c295f1e763\") " pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:02:17.022418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:17.022401 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66m5k" Jul 07 15:02:17.025509 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:17.025493 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfp7w" Jul 07 15:02:17.029846 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:17.029829 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bchjv\"" Jul 07 15:02:17.037581 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:17.037556 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:02:17.181829 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:17.181801 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-66m5k"] Jul 07 15:02:17.185977 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:02:17.185950 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae612123_dc47_43fd_b0aa_93d175975cd6.slice/crio-ca7f405febf4ca3532c4234b1137d0fa21f74a86a5add7089e3350694a25150a WatchSource:0}: Error finding container ca7f405febf4ca3532c4234b1137d0fa21f74a86a5add7089e3350694a25150a: Status 404 returned error can't find the container with id ca7f405febf4ca3532c4234b1137d0fa21f74a86a5add7089e3350694a25150a Jul 07 15:02:17.211033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:17.210999 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j52v9"] Jul 07 15:02:17.213659 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:17.213640 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kfp7w"] Jul 07 15:02:17.214335 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:02:17.214308 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b6865e3_214f_4260_87bd_66c295f1e763.slice/crio-dde29846f96e1cdbafa96232fe897da634481655a0ab248a23dc6287e63dc2f6 WatchSource:0}: Error finding container dde29846f96e1cdbafa96232fe897da634481655a0ab248a23dc6287e63dc2f6: Status 404 returned error can't find the container with id dde29846f96e1cdbafa96232fe897da634481655a0ab248a23dc6287e63dc2f6 Jul 07 15:02:17.215521 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:02:17.215496 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dece03e_eb3a_4661_855f_f37fca6d1923.slice/crio-c65d9585bc918f52799bd67c03a70f4cd3b1520593d3ef61ae1976ff96bf505a WatchSource:0}: Error finding container c65d9585bc918f52799bd67c03a70f4cd3b1520593d3ef61ae1976ff96bf505a: Status 404 returned error can't find the container with id c65d9585bc918f52799bd67c03a70f4cd3b1520593d3ef61ae1976ff96bf505a Jul 07 15:02:17.571415 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:17.571375 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kfp7w" event={"ID":"3dece03e-eb3a-4661-855f-f37fca6d1923","Type":"ContainerStarted","Data":"c65d9585bc918f52799bd67c03a70f4cd3b1520593d3ef61ae1976ff96bf505a"} Jul 07 15:02:17.572521 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:17.572501 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j52v9" event={"ID":"8b6865e3-214f-4260-87bd-66c295f1e763","Type":"ContainerStarted","Data":"dde29846f96e1cdbafa96232fe897da634481655a0ab248a23dc6287e63dc2f6"} Jul 07 15:02:17.573546 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:17.573526 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-66m5k" event={"ID":"ae612123-dc47-43fd-b0aa-93d175975cd6","Type":"ContainerStarted","Data":"ca7f405febf4ca3532c4234b1137d0fa21f74a86a5add7089e3350694a25150a"} Jul 07 15:02:20.590982 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:20.590922 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kfp7w" event={"ID":"3dece03e-eb3a-4661-855f-f37fca6d1923","Type":"ContainerStarted","Data":"3a842ac25c1d37c53adffdab1899f2f332d9a24ae9fcc695420dbd58d9179ddc"} Jul 07 15:02:21.572794 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:21.572759 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-67b5f99c8c-k666l" Jul 07 15:02:22.600840 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:22.600803 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j52v9" event={"ID":"8b6865e3-214f-4260-87bd-66c295f1e763","Type":"ContainerStarted","Data":"876932551aa8764f224fe5ae7e6ee3c5f1d2a5238e8ca64de2a4e463ff92be04"} Jul 07 15:02:22.601302 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:22.600900 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:02:22.602331 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:22.602307 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-66m5k" event={"ID":"ae612123-dc47-43fd-b0aa-93d175975cd6","Type":"ContainerStarted","Data":"f2dce11c18ed13c5a8652fc2b6f5fc384aec3b9d5f98e163393facc08425fb69"} Jul 07 15:02:22.603962 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:22.603923 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kfp7w" event={"ID":"3dece03e-eb3a-4661-855f-f37fca6d1923","Type":"ContainerStarted","Data":"c70a861e9c4d42107e2cedb2fa35228f999809fea37c9dfd46a73dda8cddd93b"} Jul 07 15:02:22.619417 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:22.619373 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-j52v9" podStartSLOduration=66.717265548 podStartE2EDuration="1m11.619359591s" totalImagesPullingTime="4.902094043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:22.618526221 +0000 UTC m=+72.043914611" watchObservedRunningTime="2026-07-07 15:02:22.619359591 +0000 UTC m=+72.044747956" Jul 07 15:02:22.635430 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:22.635378 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kfp7w" podStartSLOduration=69.276071181 podStartE2EDuration="1m11.635360472s" totalImagesPullingTime="2.359289291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:22.63504729 +0000 UTC m=+72.060435681" watchObservedRunningTime="2026-07-07 15:02:22.635360472 +0000 UTC m=+72.060748871" Jul 07 15:02:22.652251 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:22.652211 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-66m5k" podStartSLOduration=66.723656538 podStartE2EDuration="1m11.652198797s" totalImagesPullingTime="4.928542259s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:01:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:22.650822817 +0000 UTC m=+72.076211205" watchObservedRunningTime="2026-07-07 15:02:22.652198797 +0000 UTC m=+72.077587184" Jul 07 15:02:23.193796 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:23.193760 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:23.194002 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:23.193821 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:23.198594 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:23.198569 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:23.611539 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:23.611460 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:02:23.679647 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:23.679611 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5585cf8d45-h6p5f"] Jul 07 15:02:31.890986 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:31.890927 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:31.891453 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:31.891011 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:34.665090 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:34.665037 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-58755c6c75-4dgzc" podUID="cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a" containerName="console" containerID="cri-o://9d58162aeb47a6827f3c47f10f3d4c5ace79e1f7024cc6a71049c183e1316af2" gracePeriod=15 Jul 07 15:02:34.898862 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:34.898840 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:02:35.052293 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.052204 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-config\") pod \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " Jul 07 15:02:35.052293 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.052244 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-service-ca\") pod \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " Jul 07 15:02:35.052293 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.052270 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-oauth-config\") pod \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " Jul 07 15:02:35.052560 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.052314 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfkfs\" (UniqueName: \"kubernetes.io/projected/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-kube-api-access-dfkfs\") pod \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " Jul 07 15:02:35.052560 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.052351 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-serving-cert\") pod \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " Jul 07 15:02:35.052560 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.052391 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-oauth-serving-cert\") pod \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\" (UID: \"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a\") " Jul 07 15:02:35.052704 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.052675 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-config" (OuterVolumeSpecName: "console-config") pod "cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a" (UID: "cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:02:35.052816 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.052683 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-service-ca" (OuterVolumeSpecName: "service-ca") pod "cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a" (UID: "cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:02:35.052905 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.052885 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a" (UID: "cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:02:35.054789 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.054760 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-kube-api-access-dfkfs" (OuterVolumeSpecName: "kube-api-access-dfkfs") pod "cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a" (UID: "cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a"). InnerVolumeSpecName "kube-api-access-dfkfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:02:35.054789 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.054773 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a" (UID: "cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:02:35.054952 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.054833 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a" (UID: "cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:02:35.153909 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.153870 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-config\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:02:35.153909 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.153899 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-service-ca\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:02:35.153909 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.153912 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-oauth-config\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:02:35.154156 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.153924 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfkfs\" (UniqueName: \"kubernetes.io/projected/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-kube-api-access-dfkfs\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:02:35.154156 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.153957 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-console-serving-cert\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:02:35.154156 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.153970 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a-oauth-serving-cert\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:02:35.653703 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.653665 2565 generic.go:356] "Generic (PLEG): container finished" podID="cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a" containerID="9d58162aeb47a6827f3c47f10f3d4c5ace79e1f7024cc6a71049c183e1316af2" exitCode=0 Jul 07 15:02:35.653870 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.653717 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58755c6c75-4dgzc" Jul 07 15:02:35.653870 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.653757 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58755c6c75-4dgzc" event={"ID":"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a","Type":"ContainerDied","Data":"9d58162aeb47a6827f3c47f10f3d4c5ace79e1f7024cc6a71049c183e1316af2"} Jul 07 15:02:35.653870 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.653811 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58755c6c75-4dgzc" event={"ID":"cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a","Type":"ContainerDied","Data":"5834bc23f3d156c061936ec4aaf365e9aaac1061f3ef38e977e3f15b4abb60fb"} Jul 07 15:02:35.653870 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.653832 2565 scope.go:117] "RemoveContainer" containerID="9d58162aeb47a6827f3c47f10f3d4c5ace79e1f7024cc6a71049c183e1316af2" Jul 07 15:02:35.662182 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.662165 2565 scope.go:117] "RemoveContainer" containerID="9d58162aeb47a6827f3c47f10f3d4c5ace79e1f7024cc6a71049c183e1316af2" Jul 07 15:02:35.662443 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:02:35.662420 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d58162aeb47a6827f3c47f10f3d4c5ace79e1f7024cc6a71049c183e1316af2\": container with ID starting with 9d58162aeb47a6827f3c47f10f3d4c5ace79e1f7024cc6a71049c183e1316af2 not found: ID does not exist" containerID="9d58162aeb47a6827f3c47f10f3d4c5ace79e1f7024cc6a71049c183e1316af2" Jul 07 15:02:35.662538 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.662449 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d58162aeb47a6827f3c47f10f3d4c5ace79e1f7024cc6a71049c183e1316af2"} err="failed to get container status \"9d58162aeb47a6827f3c47f10f3d4c5ace79e1f7024cc6a71049c183e1316af2\": rpc error: code = NotFound desc = could not find container \"9d58162aeb47a6827f3c47f10f3d4c5ace79e1f7024cc6a71049c183e1316af2\": container with ID starting with 9d58162aeb47a6827f3c47f10f3d4c5ace79e1f7024cc6a71049c183e1316af2 not found: ID does not exist" Jul 07 15:02:35.685267 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.685238 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58755c6c75-4dgzc"] Jul 07 15:02:35.686308 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:35.686290 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58755c6c75-4dgzc"] Jul 07 15:02:37.103261 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:37.103226 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a" path="/var/lib/kubelet/pods/cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a/volumes" Jul 07 15:02:48.700182 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.700141 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5585cf8d45-h6p5f" podUID="097f8a79-ba3b-493e-8c0e-c1ae6444ca07" containerName="console" containerID="cri-o://20b26e68d816765383f3e17eeba777b888ddc3288ba3534eee41db54e0d6237c" gracePeriod=15 Jul 07 15:02:48.932224 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.932197 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:02:48.942139 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.942108 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-oauth-config\") pod \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " Jul 07 15:02:48.942257 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.942142 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-config\") pod \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " Jul 07 15:02:48.942257 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.942171 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-service-ca\") pod \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " Jul 07 15:02:48.942257 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.942195 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ckpz\" (UniqueName: \"kubernetes.io/projected/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-kube-api-access-9ckpz\") pod \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " Jul 07 15:02:48.942257 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.942222 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-oauth-serving-cert\") pod \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " Jul 07 15:02:48.942464 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.942264 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-serving-cert\") pod \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " Jul 07 15:02:48.942464 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.942338 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-trusted-ca-bundle\") pod \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\" (UID: \"097f8a79-ba3b-493e-8c0e-c1ae6444ca07\") " Jul 07 15:02:48.942577 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.942535 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-config" (OuterVolumeSpecName: "console-config") pod "097f8a79-ba3b-493e-8c0e-c1ae6444ca07" (UID: "097f8a79-ba3b-493e-8c0e-c1ae6444ca07"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:02:48.942661 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.942632 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-service-ca" (OuterVolumeSpecName: "service-ca") pod "097f8a79-ba3b-493e-8c0e-c1ae6444ca07" (UID: "097f8a79-ba3b-493e-8c0e-c1ae6444ca07"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:02:48.942852 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.942829 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "097f8a79-ba3b-493e-8c0e-c1ae6444ca07" (UID: "097f8a79-ba3b-493e-8c0e-c1ae6444ca07"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:02:48.942977 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.942950 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "097f8a79-ba3b-493e-8c0e-c1ae6444ca07" (UID: "097f8a79-ba3b-493e-8c0e-c1ae6444ca07"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:02:48.944523 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.944490 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "097f8a79-ba3b-493e-8c0e-c1ae6444ca07" (UID: "097f8a79-ba3b-493e-8c0e-c1ae6444ca07"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:02:48.944615 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.944549 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-kube-api-access-9ckpz" (OuterVolumeSpecName: "kube-api-access-9ckpz") pod "097f8a79-ba3b-493e-8c0e-c1ae6444ca07" (UID: "097f8a79-ba3b-493e-8c0e-c1ae6444ca07"). InnerVolumeSpecName "kube-api-access-9ckpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:02:48.944615 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:48.944580 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "097f8a79-ba3b-493e-8c0e-c1ae6444ca07" (UID: "097f8a79-ba3b-493e-8c0e-c1ae6444ca07"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:02:49.043092 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.042998 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-serving-cert\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:02:49.043092 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.043030 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-trusted-ca-bundle\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:02:49.043092 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.043039 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-oauth-config\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:02:49.043092 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.043048 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-console-config\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:02:49.043092 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.043056 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-service-ca\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:02:49.043092 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.043064 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ckpz\" (UniqueName: \"kubernetes.io/projected/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-kube-api-access-9ckpz\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:02:49.043092 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.043072 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/097f8a79-ba3b-493e-8c0e-c1ae6444ca07-oauth-serving-cert\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:02:49.707527 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.707485 2565 generic.go:356] "Generic (PLEG): container finished" podID="097f8a79-ba3b-493e-8c0e-c1ae6444ca07" containerID="20b26e68d816765383f3e17eeba777b888ddc3288ba3534eee41db54e0d6237c" exitCode=0 Jul 07 15:02:49.708033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.707546 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5585cf8d45-h6p5f" Jul 07 15:02:49.708033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.707566 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5585cf8d45-h6p5f" event={"ID":"097f8a79-ba3b-493e-8c0e-c1ae6444ca07","Type":"ContainerDied","Data":"20b26e68d816765383f3e17eeba777b888ddc3288ba3534eee41db54e0d6237c"} Jul 07 15:02:49.708033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.707646 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5585cf8d45-h6p5f" event={"ID":"097f8a79-ba3b-493e-8c0e-c1ae6444ca07","Type":"ContainerDied","Data":"f7d156314f497ab221d6fb6e7e57d9675ff36caa6ff12c06ca7c6ede1d4111f8"} Jul 07 15:02:49.708033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.707667 2565 scope.go:117] "RemoveContainer" containerID="20b26e68d816765383f3e17eeba777b888ddc3288ba3534eee41db54e0d6237c" Jul 07 15:02:49.715554 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.715534 2565 scope.go:117] "RemoveContainer" containerID="20b26e68d816765383f3e17eeba777b888ddc3288ba3534eee41db54e0d6237c" Jul 07 15:02:49.715785 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:02:49.715766 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20b26e68d816765383f3e17eeba777b888ddc3288ba3534eee41db54e0d6237c\": container with ID starting with 20b26e68d816765383f3e17eeba777b888ddc3288ba3534eee41db54e0d6237c not found: ID does not exist" containerID="20b26e68d816765383f3e17eeba777b888ddc3288ba3534eee41db54e0d6237c" Jul 07 15:02:49.715857 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.715795 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20b26e68d816765383f3e17eeba777b888ddc3288ba3534eee41db54e0d6237c"} err="failed to get container status \"20b26e68d816765383f3e17eeba777b888ddc3288ba3534eee41db54e0d6237c\": rpc error: code = NotFound desc = could not find container \"20b26e68d816765383f3e17eeba777b888ddc3288ba3534eee41db54e0d6237c\": container with ID starting with 20b26e68d816765383f3e17eeba777b888ddc3288ba3534eee41db54e0d6237c not found: ID does not exist" Jul 07 15:02:49.727875 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.727852 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5585cf8d45-h6p5f"] Jul 07 15:02:49.734295 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:49.734274 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5585cf8d45-h6p5f"] Jul 07 15:02:51.102231 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:51.102200 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="097f8a79-ba3b-493e-8c0e-c1ae6444ca07" path="/var/lib/kubelet/pods/097f8a79-ba3b-493e-8c0e-c1ae6444ca07/volumes" Jul 07 15:02:51.895764 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:51.895727 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:51.899684 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:51.899667 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7bfd95bbff-dq4f5" Jul 07 15:02:53.610308 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:02:53.610276 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-j52v9" Jul 07 15:03:31.751014 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.750982 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-56485b6f5-ksg6v"] Jul 07 15:03:31.751486 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.751247 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a" containerName="console" Jul 07 15:03:31.751486 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.751257 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a" containerName="console" Jul 07 15:03:31.751486 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.751282 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="097f8a79-ba3b-493e-8c0e-c1ae6444ca07" containerName="console" Jul 07 15:03:31.751486 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.751287 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="097f8a79-ba3b-493e-8c0e-c1ae6444ca07" containerName="console" Jul 07 15:03:31.751486 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.751336 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="097f8a79-ba3b-493e-8c0e-c1ae6444ca07" containerName="console" Jul 07 15:03:31.751486 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.751345 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc6ee37f-e261-4fa0-9fa1-a035b1bcd63a" containerName="console" Jul 07 15:03:31.753262 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.753243 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.756070 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.756051 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 15:03:31.757086 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.757070 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 15:03:31.757187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.757113 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 15:03:31.757248 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.757212 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 15:03:31.758161 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.758148 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pl2ff\"" Jul 07 15:03:31.758552 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.758523 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 15:03:31.771261 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.771225 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 15:03:31.772654 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.772633 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-56485b6f5-ksg6v"] Jul 07 15:03:31.879659 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.879625 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29ba6248-6819-457a-b65a-6ba748d86a0a-metrics-client-ca\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.879826 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.879682 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29ba6248-6819-457a-b65a-6ba748d86a0a-serving-certs-ca-bundle\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.879826 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.879751 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/29ba6248-6819-457a-b65a-6ba748d86a0a-secret-telemeter-client\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.879826 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.879785 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29ba6248-6819-457a-b65a-6ba748d86a0a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.879826 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.879808 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/29ba6248-6819-457a-b65a-6ba748d86a0a-telemeter-client-tls\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.879982 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.879833 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29ba6248-6819-457a-b65a-6ba748d86a0a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.879982 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.879876 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/29ba6248-6819-457a-b65a-6ba748d86a0a-federate-client-tls\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.879982 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.879897 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5pw9\" (UniqueName: \"kubernetes.io/projected/29ba6248-6819-457a-b65a-6ba748d86a0a-kube-api-access-j5pw9\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.980677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.980634 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/29ba6248-6819-457a-b65a-6ba748d86a0a-secret-telemeter-client\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.980677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.980674 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29ba6248-6819-457a-b65a-6ba748d86a0a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.980933 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.980693 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/29ba6248-6819-457a-b65a-6ba748d86a0a-telemeter-client-tls\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.980933 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.980731 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29ba6248-6819-457a-b65a-6ba748d86a0a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.980933 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.980764 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/29ba6248-6819-457a-b65a-6ba748d86a0a-federate-client-tls\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.980933 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.980788 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5pw9\" (UniqueName: \"kubernetes.io/projected/29ba6248-6819-457a-b65a-6ba748d86a0a-kube-api-access-j5pw9\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.980933 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.980833 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29ba6248-6819-457a-b65a-6ba748d86a0a-metrics-client-ca\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.980933 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.980914 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29ba6248-6819-457a-b65a-6ba748d86a0a-serving-certs-ca-bundle\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.981716 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.981685 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29ba6248-6819-457a-b65a-6ba748d86a0a-serving-certs-ca-bundle\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.981716 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.981701 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29ba6248-6819-457a-b65a-6ba748d86a0a-metrics-client-ca\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.981862 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.981752 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29ba6248-6819-457a-b65a-6ba748d86a0a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.983571 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.983545 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/29ba6248-6819-457a-b65a-6ba748d86a0a-federate-client-tls\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.983683 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.983664 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/29ba6248-6819-457a-b65a-6ba748d86a0a-telemeter-client-tls\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.983741 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.983722 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/29ba6248-6819-457a-b65a-6ba748d86a0a-secret-telemeter-client\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.983779 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.983737 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29ba6248-6819-457a-b65a-6ba748d86a0a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:31.990722 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:31.990701 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5pw9\" (UniqueName: \"kubernetes.io/projected/29ba6248-6819-457a-b65a-6ba748d86a0a-kube-api-access-j5pw9\") pod \"telemeter-client-56485b6f5-ksg6v\" (UID: \"29ba6248-6819-457a-b65a-6ba748d86a0a\") " pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:32.062449 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:32.062370 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" Jul 07 15:03:32.194260 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:32.194222 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-56485b6f5-ksg6v"] Jul 07 15:03:32.197345 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:03:32.197317 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ba6248_6819_457a_b65a_6ba748d86a0a.slice/crio-2855f04857762174b1fec09b2eacb03c83b2777b33d95034a650ab959f352a36 WatchSource:0}: Error finding container 2855f04857762174b1fec09b2eacb03c83b2777b33d95034a650ab959f352a36: Status 404 returned error can't find the container with id 2855f04857762174b1fec09b2eacb03c83b2777b33d95034a650ab959f352a36 Jul 07 15:03:32.866441 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:32.866400 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" event={"ID":"29ba6248-6819-457a-b65a-6ba748d86a0a","Type":"ContainerStarted","Data":"2855f04857762174b1fec09b2eacb03c83b2777b33d95034a650ab959f352a36"} Jul 07 15:03:34.874709 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:34.874672 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" event={"ID":"29ba6248-6819-457a-b65a-6ba748d86a0a","Type":"ContainerStarted","Data":"1e9a387ca5ca49a01d3370e4140cd1cdb1642a9d85fb5f2eff763e7a69ea9738"} Jul 07 15:03:37.890480 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:37.890437 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" event={"ID":"29ba6248-6819-457a-b65a-6ba748d86a0a","Type":"ContainerStarted","Data":"bc2589ca24cc6cc0435c4eb098c45ed5400fc63eab4c1ae1b69c651d04ebd12d"} Jul 07 15:03:37.890884 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:37.890507 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" event={"ID":"29ba6248-6819-457a-b65a-6ba748d86a0a","Type":"ContainerStarted","Data":"5cf331d09d075feaa50e1f1b9034e9a808a5bc32308ecbdfb17694618699bc47"} Jul 07 15:03:37.923006 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:37.922922 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-56485b6f5-ksg6v" podStartSLOduration=1.897384763 podStartE2EDuration="6.92289644s" totalImagesPullingTime="5.025511677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:03:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:03:37.921187884 +0000 UTC m=+147.346576274" watchObservedRunningTime="2026-07-07 15:03:37.92289644 +0000 UTC m=+147.348284828" Jul 07 15:03:38.689703 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.689667 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74c6c56968-9gzn8"] Jul 07 15:03:38.691974 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.691929 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.710141 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.710108 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74c6c56968-9gzn8"] Jul 07 15:03:38.842198 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.842151 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-oauth-serving-cert\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.842198 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.842209 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-config\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.842420 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.842239 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qjzs\" (UniqueName: \"kubernetes.io/projected/c501b48b-6d4a-4c03-89d1-18a40edf434a-kube-api-access-2qjzs\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.842420 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.842267 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-trusted-ca-bundle\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.842420 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.842297 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-service-ca\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.842420 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.842328 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-oauth-config\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.842420 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.842381 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-serving-cert\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.943725 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.943640 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-serving-cert\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.943725 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.943680 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-oauth-serving-cert\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.943725 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.943696 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-config\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.943725 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.943717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qjzs\" (UniqueName: \"kubernetes.io/projected/c501b48b-6d4a-4c03-89d1-18a40edf434a-kube-api-access-2qjzs\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.944363 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.943739 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-trusted-ca-bundle\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.944363 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.943793 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-service-ca\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.944363 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.943836 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-oauth-config\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.944579 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.944554 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-oauth-serving-cert\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.944635 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.944556 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-config\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.944635 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.944605 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-service-ca\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.944635 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.944619 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-trusted-ca-bundle\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.946161 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.946142 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-serving-cert\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.946264 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.946202 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-oauth-config\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:38.954411 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:38.954392 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qjzs\" (UniqueName: \"kubernetes.io/projected/c501b48b-6d4a-4c03-89d1-18a40edf434a-kube-api-access-2qjzs\") pod \"console-74c6c56968-9gzn8\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:39.000838 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:39.000810 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:39.129349 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:39.129331 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74c6c56968-9gzn8"] Jul 07 15:03:39.131976 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:03:39.131929 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc501b48b_6d4a_4c03_89d1_18a40edf434a.slice/crio-b1948ef2ddeefa9d68cd518f9ec0858bb26170c8ba4a27e78a091b3bca94016e WatchSource:0}: Error finding container b1948ef2ddeefa9d68cd518f9ec0858bb26170c8ba4a27e78a091b3bca94016e: Status 404 returned error can't find the container with id b1948ef2ddeefa9d68cd518f9ec0858bb26170c8ba4a27e78a091b3bca94016e Jul 07 15:03:39.899211 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:39.899172 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74c6c56968-9gzn8" event={"ID":"c501b48b-6d4a-4c03-89d1-18a40edf434a","Type":"ContainerStarted","Data":"334597a434d1ec714868b100a72b29d173234886e5d74a125feef1692df29886"} Jul 07 15:03:39.899395 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:39.899221 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74c6c56968-9gzn8" event={"ID":"c501b48b-6d4a-4c03-89d1-18a40edf434a","Type":"ContainerStarted","Data":"b1948ef2ddeefa9d68cd518f9ec0858bb26170c8ba4a27e78a091b3bca94016e"} Jul 07 15:03:39.920075 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:39.920022 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74c6c56968-9gzn8" podStartSLOduration=1.920002698 podStartE2EDuration="1.920002698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:03:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:03:39.919193029 +0000 UTC m=+149.344581419" watchObservedRunningTime="2026-07-07 15:03:39.920002698 +0000 UTC m=+149.345391087" Jul 07 15:03:44.357650 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.357619 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74c6c56968-9gzn8"] Jul 07 15:03:44.390846 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.390819 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-87747b946-zsdv6"] Jul 07 15:03:44.393110 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.393093 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.405256 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.405235 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-87747b946-zsdv6"] Jul 07 15:03:44.586504 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.586465 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hfpq\" (UniqueName: \"kubernetes.io/projected/50783695-e692-4b13-9108-0ff7e112a892-kube-api-access-7hfpq\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.586504 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.586512 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50783695-e692-4b13-9108-0ff7e112a892-console-oauth-config\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.586738 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.586568 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50783695-e692-4b13-9108-0ff7e112a892-service-ca\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.586738 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.586620 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50783695-e692-4b13-9108-0ff7e112a892-console-config\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.586738 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.586663 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50783695-e692-4b13-9108-0ff7e112a892-oauth-serving-cert\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.586864 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.586731 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50783695-e692-4b13-9108-0ff7e112a892-trusted-ca-bundle\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.586864 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.586764 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50783695-e692-4b13-9108-0ff7e112a892-console-serving-cert\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.687898 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.687864 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hfpq\" (UniqueName: \"kubernetes.io/projected/50783695-e692-4b13-9108-0ff7e112a892-kube-api-access-7hfpq\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.688112 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.687906 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50783695-e692-4b13-9108-0ff7e112a892-console-oauth-config\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.688112 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.687931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50783695-e692-4b13-9108-0ff7e112a892-service-ca\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.688112 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.687994 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50783695-e692-4b13-9108-0ff7e112a892-console-config\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.688112 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.688035 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50783695-e692-4b13-9108-0ff7e112a892-oauth-serving-cert\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.688112 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.688079 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50783695-e692-4b13-9108-0ff7e112a892-trusted-ca-bundle\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.688112 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.688107 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50783695-e692-4b13-9108-0ff7e112a892-console-serving-cert\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.688877 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.688840 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50783695-e692-4b13-9108-0ff7e112a892-console-config\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.688877 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.688871 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50783695-e692-4b13-9108-0ff7e112a892-service-ca\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.689085 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.688983 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50783695-e692-4b13-9108-0ff7e112a892-trusted-ca-bundle\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.689085 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.688983 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50783695-e692-4b13-9108-0ff7e112a892-oauth-serving-cert\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.690549 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.690524 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50783695-e692-4b13-9108-0ff7e112a892-console-oauth-config\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.690631 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.690521 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50783695-e692-4b13-9108-0ff7e112a892-console-serving-cert\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.697125 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.697101 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hfpq\" (UniqueName: \"kubernetes.io/projected/50783695-e692-4b13-9108-0ff7e112a892-kube-api-access-7hfpq\") pod \"console-87747b946-zsdv6\" (UID: \"50783695-e692-4b13-9108-0ff7e112a892\") " pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.701877 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.701856 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:44.844784 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.844760 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-87747b946-zsdv6"] Jul 07 15:03:44.847175 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:03:44.847151 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50783695_e692_4b13_9108_0ff7e112a892.slice/crio-ede10ab618f7692c18fac70a6631f7e95d4e2adfeb43b13ae4c4c725089fa710 WatchSource:0}: Error finding container ede10ab618f7692c18fac70a6631f7e95d4e2adfeb43b13ae4c4c725089fa710: Status 404 returned error can't find the container with id ede10ab618f7692c18fac70a6631f7e95d4e2adfeb43b13ae4c4c725089fa710 Jul 07 15:03:44.918132 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.918093 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-87747b946-zsdv6" event={"ID":"50783695-e692-4b13-9108-0ff7e112a892","Type":"ContainerStarted","Data":"970bf14d71563b76285dae2a75cc5b49c493a763a59de327c82ea55529daa462"} Jul 07 15:03:44.918277 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.918144 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-87747b946-zsdv6" event={"ID":"50783695-e692-4b13-9108-0ff7e112a892","Type":"ContainerStarted","Data":"ede10ab618f7692c18fac70a6631f7e95d4e2adfeb43b13ae4c4c725089fa710"} Jul 07 15:03:44.939214 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:44.939103 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-87747b946-zsdv6" podStartSLOduration=0.939086589 podStartE2EDuration="939.086589ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:03:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:03:44.938158402 +0000 UTC m=+154.363546790" watchObservedRunningTime="2026-07-07 15:03:44.939086589 +0000 UTC m=+154.364475014" Jul 07 15:03:49.001413 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:49.001375 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:03:54.702520 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:54.702485 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:54.702908 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:54.702552 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:54.707138 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:54.707116 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:54.961887 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:54.961798 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-87747b946-zsdv6" Jul 07 15:03:55.010769 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:03:55.010738 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-95569c898-4w6xm"] Jul 07 15:04:09.377575 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.377487 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-74c6c56968-9gzn8" podUID="c501b48b-6d4a-4c03-89d1-18a40edf434a" containerName="console" containerID="cri-o://334597a434d1ec714868b100a72b29d173234886e5d74a125feef1692df29886" gracePeriod=15 Jul 07 15:04:09.623099 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.623076 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:04:09.691232 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.691196 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-oauth-config\") pod \"c501b48b-6d4a-4c03-89d1-18a40edf434a\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " Jul 07 15:04:09.691418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.691250 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-oauth-serving-cert\") pod \"c501b48b-6d4a-4c03-89d1-18a40edf434a\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " Jul 07 15:04:09.691418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.691308 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-serving-cert\") pod \"c501b48b-6d4a-4c03-89d1-18a40edf434a\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " Jul 07 15:04:09.691418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.691335 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-trusted-ca-bundle\") pod \"c501b48b-6d4a-4c03-89d1-18a40edf434a\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " Jul 07 15:04:09.691418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.691357 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-service-ca\") pod \"c501b48b-6d4a-4c03-89d1-18a40edf434a\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " Jul 07 15:04:09.691418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.691383 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-config\") pod \"c501b48b-6d4a-4c03-89d1-18a40edf434a\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " Jul 07 15:04:09.691655 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.691427 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qjzs\" (UniqueName: \"kubernetes.io/projected/c501b48b-6d4a-4c03-89d1-18a40edf434a-kube-api-access-2qjzs\") pod \"c501b48b-6d4a-4c03-89d1-18a40edf434a\" (UID: \"c501b48b-6d4a-4c03-89d1-18a40edf434a\") " Jul 07 15:04:09.691794 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.691757 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c501b48b-6d4a-4c03-89d1-18a40edf434a" (UID: "c501b48b-6d4a-4c03-89d1-18a40edf434a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:04:09.691794 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.691782 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c501b48b-6d4a-4c03-89d1-18a40edf434a" (UID: "c501b48b-6d4a-4c03-89d1-18a40edf434a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:04:09.691912 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.691815 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-config" (OuterVolumeSpecName: "console-config") pod "c501b48b-6d4a-4c03-89d1-18a40edf434a" (UID: "c501b48b-6d4a-4c03-89d1-18a40edf434a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:04:09.691912 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.691829 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-service-ca" (OuterVolumeSpecName: "service-ca") pod "c501b48b-6d4a-4c03-89d1-18a40edf434a" (UID: "c501b48b-6d4a-4c03-89d1-18a40edf434a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:04:09.693550 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.693522 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c501b48b-6d4a-4c03-89d1-18a40edf434a" (UID: "c501b48b-6d4a-4c03-89d1-18a40edf434a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:04:09.693721 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.693565 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c501b48b-6d4a-4c03-89d1-18a40edf434a" (UID: "c501b48b-6d4a-4c03-89d1-18a40edf434a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:04:09.693721 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.693622 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c501b48b-6d4a-4c03-89d1-18a40edf434a-kube-api-access-2qjzs" (OuterVolumeSpecName: "kube-api-access-2qjzs") pod "c501b48b-6d4a-4c03-89d1-18a40edf434a" (UID: "c501b48b-6d4a-4c03-89d1-18a40edf434a"). InnerVolumeSpecName "kube-api-access-2qjzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:04:09.793021 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.792981 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qjzs\" (UniqueName: \"kubernetes.io/projected/c501b48b-6d4a-4c03-89d1-18a40edf434a-kube-api-access-2qjzs\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:09.793021 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.793016 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-oauth-config\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:09.793021 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.793028 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-oauth-serving-cert\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:09.793260 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.793040 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-serving-cert\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:09.793260 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.793051 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-trusted-ca-bundle\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:09.793260 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.793063 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-service-ca\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:09.793260 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:09.793074 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c501b48b-6d4a-4c03-89d1-18a40edf434a-console-config\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:10.014699 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:10.014603 2565 generic.go:356] "Generic (PLEG): container finished" podID="c501b48b-6d4a-4c03-89d1-18a40edf434a" containerID="334597a434d1ec714868b100a72b29d173234886e5d74a125feef1692df29886" exitCode=0 Jul 07 15:04:10.014699 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:10.014653 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74c6c56968-9gzn8" event={"ID":"c501b48b-6d4a-4c03-89d1-18a40edf434a","Type":"ContainerDied","Data":"334597a434d1ec714868b100a72b29d173234886e5d74a125feef1692df29886"} Jul 07 15:04:10.014699 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:10.014682 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74c6c56968-9gzn8" Jul 07 15:04:10.014929 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:10.014734 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74c6c56968-9gzn8" event={"ID":"c501b48b-6d4a-4c03-89d1-18a40edf434a","Type":"ContainerDied","Data":"b1948ef2ddeefa9d68cd518f9ec0858bb26170c8ba4a27e78a091b3bca94016e"} Jul 07 15:04:10.014929 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:10.014761 2565 scope.go:117] "RemoveContainer" containerID="334597a434d1ec714868b100a72b29d173234886e5d74a125feef1692df29886" Jul 07 15:04:10.031495 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:10.031466 2565 scope.go:117] "RemoveContainer" containerID="334597a434d1ec714868b100a72b29d173234886e5d74a125feef1692df29886" Jul 07 15:04:10.031820 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:04:10.031799 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"334597a434d1ec714868b100a72b29d173234886e5d74a125feef1692df29886\": container with ID starting with 334597a434d1ec714868b100a72b29d173234886e5d74a125feef1692df29886 not found: ID does not exist" containerID="334597a434d1ec714868b100a72b29d173234886e5d74a125feef1692df29886" Jul 07 15:04:10.031890 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:10.031830 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"334597a434d1ec714868b100a72b29d173234886e5d74a125feef1692df29886"} err="failed to get container status \"334597a434d1ec714868b100a72b29d173234886e5d74a125feef1692df29886\": rpc error: code = NotFound desc = could not find container \"334597a434d1ec714868b100a72b29d173234886e5d74a125feef1692df29886\": container with ID starting with 334597a434d1ec714868b100a72b29d173234886e5d74a125feef1692df29886 not found: ID does not exist" Jul 07 15:04:10.050997 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:10.050959 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74c6c56968-9gzn8"] Jul 07 15:04:10.059853 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:10.059825 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74c6c56968-9gzn8"] Jul 07 15:04:11.103411 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:11.103378 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c501b48b-6d4a-4c03-89d1-18a40edf434a" path="/var/lib/kubelet/pods/c501b48b-6d4a-4c03-89d1-18a40edf434a/volumes" Jul 07 15:04:20.032875 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.032808 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-95569c898-4w6xm" podUID="8bf0efcb-6b99-4e92-ae58-bdf839edde2b" containerName="console" containerID="cri-o://7c9a827a90004e0d2793b94f92a70a4ae1be0dda5ab6dd1a0438583b0e314c1c" gracePeriod=15 Jul 07 15:04:20.280591 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.280567 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:04:20.383752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.383669 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-847r4\" (UniqueName: \"kubernetes.io/projected/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-kube-api-access-847r4\") pod \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " Jul 07 15:04:20.383752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.383714 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-trusted-ca-bundle\") pod \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " Jul 07 15:04:20.383752 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.383737 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-oauth-serving-cert\") pod \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " Jul 07 15:04:20.384033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.383760 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-config\") pod \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " Jul 07 15:04:20.384033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.383785 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-serving-cert\") pod \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " Jul 07 15:04:20.384033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.383851 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-service-ca\") pod \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " Jul 07 15:04:20.384033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.383889 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-oauth-config\") pod \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\" (UID: \"8bf0efcb-6b99-4e92-ae58-bdf839edde2b\") " Jul 07 15:04:20.384228 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.384200 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-config" (OuterVolumeSpecName: "console-config") pod "8bf0efcb-6b99-4e92-ae58-bdf839edde2b" (UID: "8bf0efcb-6b99-4e92-ae58-bdf839edde2b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:04:20.384228 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.384211 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8bf0efcb-6b99-4e92-ae58-bdf839edde2b" (UID: "8bf0efcb-6b99-4e92-ae58-bdf839edde2b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:04:20.384569 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.384539 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-service-ca" (OuterVolumeSpecName: "service-ca") pod "8bf0efcb-6b99-4e92-ae58-bdf839edde2b" (UID: "8bf0efcb-6b99-4e92-ae58-bdf839edde2b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:04:20.384673 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.384546 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8bf0efcb-6b99-4e92-ae58-bdf839edde2b" (UID: "8bf0efcb-6b99-4e92-ae58-bdf839edde2b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:04:20.386036 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.386008 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-kube-api-access-847r4" (OuterVolumeSpecName: "kube-api-access-847r4") pod "8bf0efcb-6b99-4e92-ae58-bdf839edde2b" (UID: "8bf0efcb-6b99-4e92-ae58-bdf839edde2b"). InnerVolumeSpecName "kube-api-access-847r4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:04:20.386221 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.386194 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8bf0efcb-6b99-4e92-ae58-bdf839edde2b" (UID: "8bf0efcb-6b99-4e92-ae58-bdf839edde2b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:04:20.386305 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.386277 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8bf0efcb-6b99-4e92-ae58-bdf839edde2b" (UID: "8bf0efcb-6b99-4e92-ae58-bdf839edde2b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:04:20.485080 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.485044 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-847r4\" (UniqueName: \"kubernetes.io/projected/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-kube-api-access-847r4\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:20.485080 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.485071 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-trusted-ca-bundle\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:20.485080 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.485080 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-oauth-serving-cert\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:20.485080 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.485089 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-config\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:20.485339 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.485099 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-serving-cert\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:20.485339 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.485107 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-service-ca\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:20.485339 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:20.485116 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bf0efcb-6b99-4e92-ae58-bdf839edde2b-console-oauth-config\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:04:21.055266 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:21.055235 2565 generic.go:356] "Generic (PLEG): container finished" podID="8bf0efcb-6b99-4e92-ae58-bdf839edde2b" containerID="7c9a827a90004e0d2793b94f92a70a4ae1be0dda5ab6dd1a0438583b0e314c1c" exitCode=0 Jul 07 15:04:21.055645 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:21.055279 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-95569c898-4w6xm" Jul 07 15:04:21.055645 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:21.055321 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-95569c898-4w6xm" event={"ID":"8bf0efcb-6b99-4e92-ae58-bdf839edde2b","Type":"ContainerDied","Data":"7c9a827a90004e0d2793b94f92a70a4ae1be0dda5ab6dd1a0438583b0e314c1c"} Jul 07 15:04:21.055645 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:21.055373 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-95569c898-4w6xm" event={"ID":"8bf0efcb-6b99-4e92-ae58-bdf839edde2b","Type":"ContainerDied","Data":"6a585e1c65221550cd065002fa8ab410fdc91f20da860245bb43cc3894da91eb"} Jul 07 15:04:21.055645 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:21.055394 2565 scope.go:117] "RemoveContainer" containerID="7c9a827a90004e0d2793b94f92a70a4ae1be0dda5ab6dd1a0438583b0e314c1c" Jul 07 15:04:21.063862 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:21.063847 2565 scope.go:117] "RemoveContainer" containerID="7c9a827a90004e0d2793b94f92a70a4ae1be0dda5ab6dd1a0438583b0e314c1c" Jul 07 15:04:21.064095 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:04:21.064079 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c9a827a90004e0d2793b94f92a70a4ae1be0dda5ab6dd1a0438583b0e314c1c\": container with ID starting with 7c9a827a90004e0d2793b94f92a70a4ae1be0dda5ab6dd1a0438583b0e314c1c not found: ID does not exist" containerID="7c9a827a90004e0d2793b94f92a70a4ae1be0dda5ab6dd1a0438583b0e314c1c" Jul 07 15:04:21.064156 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:21.064102 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9a827a90004e0d2793b94f92a70a4ae1be0dda5ab6dd1a0438583b0e314c1c"} err="failed to get container status \"7c9a827a90004e0d2793b94f92a70a4ae1be0dda5ab6dd1a0438583b0e314c1c\": rpc error: code = NotFound desc = could not find container \"7c9a827a90004e0d2793b94f92a70a4ae1be0dda5ab6dd1a0438583b0e314c1c\": container with ID starting with 7c9a827a90004e0d2793b94f92a70a4ae1be0dda5ab6dd1a0438583b0e314c1c not found: ID does not exist" Jul 07 15:04:21.077079 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:21.077051 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-95569c898-4w6xm"] Jul 07 15:04:21.084029 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:21.084010 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-95569c898-4w6xm"] Jul 07 15:04:21.102202 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:21.102179 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bf0efcb-6b99-4e92-ae58-bdf839edde2b" path="/var/lib/kubelet/pods/8bf0efcb-6b99-4e92-ae58-bdf839edde2b/volumes" Jul 07 15:04:41.498033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.497991 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s"] Jul 07 15:04:41.498533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.498284 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bf0efcb-6b99-4e92-ae58-bdf839edde2b" containerName="console" Jul 07 15:04:41.498533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.498295 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf0efcb-6b99-4e92-ae58-bdf839edde2b" containerName="console" Jul 07 15:04:41.498533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.498306 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c501b48b-6d4a-4c03-89d1-18a40edf434a" containerName="console" Jul 07 15:04:41.498533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.498311 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c501b48b-6d4a-4c03-89d1-18a40edf434a" containerName="console" Jul 07 15:04:41.498533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.498363 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c501b48b-6d4a-4c03-89d1-18a40edf434a" containerName="console" Jul 07 15:04:41.498533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.498371 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bf0efcb-6b99-4e92-ae58-bdf839edde2b" containerName="console" Jul 07 15:04:41.501558 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.501541 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" Jul 07 15:04:41.504096 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.504068 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 15:04:41.504231 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.504072 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 15:04:41.504231 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.504121 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-64p5b\"" Jul 07 15:04:41.509692 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.509666 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s"] Jul 07 15:04:41.555649 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.555616 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s\" (UID: \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" Jul 07 15:04:41.555649 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.555650 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s\" (UID: \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" Jul 07 15:04:41.555865 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.555675 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r7wk\" (UniqueName: \"kubernetes.io/projected/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-kube-api-access-8r7wk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s\" (UID: \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" Jul 07 15:04:41.656582 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.656549 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s\" (UID: \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" Jul 07 15:04:41.656582 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.656582 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s\" (UID: \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" Jul 07 15:04:41.656829 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.656607 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r7wk\" (UniqueName: \"kubernetes.io/projected/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-kube-api-access-8r7wk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s\" (UID: \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" Jul 07 15:04:41.657018 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.656995 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s\" (UID: \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" Jul 07 15:04:41.657089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.657014 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s\" (UID: \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" Jul 07 15:04:41.665917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.665888 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r7wk\" (UniqueName: \"kubernetes.io/projected/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-kube-api-access-8r7wk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s\" (UID: \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" Jul 07 15:04:41.811732 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.811646 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" Jul 07 15:04:41.946158 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:41.946094 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s"] Jul 07 15:04:41.948650 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:04:41.948624 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef9fc4d3_91c6_40e8_a31a_af367daaf1c2.slice/crio-380ec25ef14f49052c71e688e87f2dfd3355ffac6cb95548870ee4b655a93bba WatchSource:0}: Error finding container 380ec25ef14f49052c71e688e87f2dfd3355ffac6cb95548870ee4b655a93bba: Status 404 returned error can't find the container with id 380ec25ef14f49052c71e688e87f2dfd3355ffac6cb95548870ee4b655a93bba Jul 07 15:04:42.133800 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:42.133701 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" event={"ID":"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2","Type":"ContainerStarted","Data":"380ec25ef14f49052c71e688e87f2dfd3355ffac6cb95548870ee4b655a93bba"} Jul 07 15:04:55.186216 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:55.186183 2565 generic.go:356] "Generic (PLEG): container finished" podID="ef9fc4d3-91c6-40e8-a31a-af367daaf1c2" containerID="261f4858289cde9a74cb89ac5bd6cf59a9be66d62e03aa4ba66897f63bd79d0e" exitCode=0 Jul 07 15:04:55.186549 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:55.186274 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" event={"ID":"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2","Type":"ContainerDied","Data":"261f4858289cde9a74cb89ac5bd6cf59a9be66d62e03aa4ba66897f63bd79d0e"} Jul 07 15:04:58.199463 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:58.199427 2565 generic.go:356] "Generic (PLEG): container finished" podID="ef9fc4d3-91c6-40e8-a31a-af367daaf1c2" containerID="38728c0eeabb9b49da6d79c189fe50f8d4f54a85fbd187a8a56377efc2f12c8e" exitCode=0 Jul 07 15:04:58.200235 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:04:58.199477 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" event={"ID":"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2","Type":"ContainerDied","Data":"38728c0eeabb9b49da6d79c189fe50f8d4f54a85fbd187a8a56377efc2f12c8e"} Jul 07 15:05:05.231783 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:05.231741 2565 generic.go:356] "Generic (PLEG): container finished" podID="ef9fc4d3-91c6-40e8-a31a-af367daaf1c2" containerID="5e2a65e7410d8961a0d00ce148619b363fd6ddd6ffd3b5f0a9f2bb88d62522ef" exitCode=0 Jul 07 15:05:05.232206 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:05.231792 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" event={"ID":"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2","Type":"ContainerDied","Data":"5e2a65e7410d8961a0d00ce148619b363fd6ddd6ffd3b5f0a9f2bb88d62522ef"} Jul 07 15:05:06.357749 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:06.357724 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" Jul 07 15:05:06.459060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:06.459022 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-bundle\") pod \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\" (UID: \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\") " Jul 07 15:05:06.459060 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:06.459069 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-util\") pod \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\" (UID: \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\") " Jul 07 15:05:06.459304 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:06.459096 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r7wk\" (UniqueName: \"kubernetes.io/projected/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-kube-api-access-8r7wk\") pod \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\" (UID: \"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2\") " Jul 07 15:05:06.459724 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:06.459701 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-bundle" (OuterVolumeSpecName: "bundle") pod "ef9fc4d3-91c6-40e8-a31a-af367daaf1c2" (UID: "ef9fc4d3-91c6-40e8-a31a-af367daaf1c2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:05:06.461286 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:06.461265 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-kube-api-access-8r7wk" (OuterVolumeSpecName: "kube-api-access-8r7wk") pod "ef9fc4d3-91c6-40e8-a31a-af367daaf1c2" (UID: "ef9fc4d3-91c6-40e8-a31a-af367daaf1c2"). InnerVolumeSpecName "kube-api-access-8r7wk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:05:06.463902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:06.463871 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-util" (OuterVolumeSpecName: "util") pod "ef9fc4d3-91c6-40e8-a31a-af367daaf1c2" (UID: "ef9fc4d3-91c6-40e8-a31a-af367daaf1c2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:05:06.560355 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:06.560242 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-util\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:05:06.560355 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:06.560289 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r7wk\" (UniqueName: \"kubernetes.io/projected/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-kube-api-access-8r7wk\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:05:06.560355 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:06.560303 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef9fc4d3-91c6-40e8-a31a-af367daaf1c2-bundle\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:05:07.241387 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:07.241358 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" Jul 07 15:05:07.241548 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:07.241352 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xtg7s" event={"ID":"ef9fc4d3-91c6-40e8-a31a-af367daaf1c2","Type":"ContainerDied","Data":"380ec25ef14f49052c71e688e87f2dfd3355ffac6cb95548870ee4b655a93bba"} Jul 07 15:05:07.241548 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:07.241472 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="380ec25ef14f49052c71e688e87f2dfd3355ffac6cb95548870ee4b655a93bba" Jul 07 15:05:14.566562 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.566529 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7"] Jul 07 15:05:14.566975 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.566816 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef9fc4d3-91c6-40e8-a31a-af367daaf1c2" containerName="util" Jul 07 15:05:14.566975 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.566826 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef9fc4d3-91c6-40e8-a31a-af367daaf1c2" containerName="util" Jul 07 15:05:14.566975 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.566841 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef9fc4d3-91c6-40e8-a31a-af367daaf1c2" containerName="pull" Jul 07 15:05:14.566975 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.566847 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef9fc4d3-91c6-40e8-a31a-af367daaf1c2" containerName="pull" Jul 07 15:05:14.566975 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.566858 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef9fc4d3-91c6-40e8-a31a-af367daaf1c2" containerName="extract" Jul 07 15:05:14.566975 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.566863 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef9fc4d3-91c6-40e8-a31a-af367daaf1c2" containerName="extract" Jul 07 15:05:14.566975 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.566909 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef9fc4d3-91c6-40e8-a31a-af367daaf1c2" containerName="extract" Jul 07 15:05:14.602664 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.602635 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7"] Jul 07 15:05:14.602809 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.602732 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7" Jul 07 15:05:14.605454 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.605433 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:05:14.605571 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.605433 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 15:05:14.605571 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.605528 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-x8zmq\"" Jul 07 15:05:14.725549 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.725520 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwtvq\" (UniqueName: \"kubernetes.io/projected/94f8251e-eb9c-450b-9905-4336c0dc7dae-kube-api-access-pwtvq\") pod \"cert-manager-operator-controller-manager-55c444bf86-qvzq7\" (UID: \"94f8251e-eb9c-450b-9905-4336c0dc7dae\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7" Jul 07 15:05:14.725729 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.725565 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94f8251e-eb9c-450b-9905-4336c0dc7dae-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qvzq7\" (UID: \"94f8251e-eb9c-450b-9905-4336c0dc7dae\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7" Jul 07 15:05:14.826880 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.826794 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94f8251e-eb9c-450b-9905-4336c0dc7dae-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qvzq7\" (UID: \"94f8251e-eb9c-450b-9905-4336c0dc7dae\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7" Jul 07 15:05:14.827034 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.826886 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwtvq\" (UniqueName: \"kubernetes.io/projected/94f8251e-eb9c-450b-9905-4336c0dc7dae-kube-api-access-pwtvq\") pod \"cert-manager-operator-controller-manager-55c444bf86-qvzq7\" (UID: \"94f8251e-eb9c-450b-9905-4336c0dc7dae\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7" Jul 07 15:05:14.827220 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.827201 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94f8251e-eb9c-450b-9905-4336c0dc7dae-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qvzq7\" (UID: \"94f8251e-eb9c-450b-9905-4336c0dc7dae\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7" Jul 07 15:05:14.840994 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.840970 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwtvq\" (UniqueName: \"kubernetes.io/projected/94f8251e-eb9c-450b-9905-4336c0dc7dae-kube-api-access-pwtvq\") pod \"cert-manager-operator-controller-manager-55c444bf86-qvzq7\" (UID: \"94f8251e-eb9c-450b-9905-4336c0dc7dae\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7" Jul 07 15:05:14.911668 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:14.911634 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7" Jul 07 15:05:15.060959 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:15.060911 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7"] Jul 07 15:05:15.063888 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:05:15.063857 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94f8251e_eb9c_450b_9905_4336c0dc7dae.slice/crio-27458d997915f238beba9153adec6af78ada9859b315cb6faa4dc34353d6dfca WatchSource:0}: Error finding container 27458d997915f238beba9153adec6af78ada9859b315cb6faa4dc34353d6dfca: Status 404 returned error can't find the container with id 27458d997915f238beba9153adec6af78ada9859b315cb6faa4dc34353d6dfca Jul 07 15:05:15.273252 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:15.273217 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7" event={"ID":"94f8251e-eb9c-450b-9905-4336c0dc7dae","Type":"ContainerStarted","Data":"27458d997915f238beba9153adec6af78ada9859b315cb6faa4dc34353d6dfca"} Jul 07 15:05:18.287474 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:18.287434 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7" event={"ID":"94f8251e-eb9c-450b-9905-4336c0dc7dae","Type":"ContainerStarted","Data":"41021220a1aaa1f35c1c15a5f1245734edd5ee121ab21d257ab4754189c11ac8"} Jul 07 15:05:18.346024 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:18.345973 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qvzq7" podStartSLOduration=2.082961633 podStartE2EDuration="4.345956303s" totalImagesPullingTime="2.26299467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:18.343590568 +0000 UTC m=+247.768978956" watchObservedRunningTime="2026-07-07 15:05:18.345956303 +0000 UTC m=+247.771344684" Jul 07 15:05:19.174170 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.174134 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6"] Jul 07 15:05:19.176617 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.176602 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" Jul 07 15:05:19.182455 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.182433 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 15:05:19.182812 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.182796 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 15:05:19.183446 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.183425 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-64p5b\"" Jul 07 15:05:19.195111 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.195091 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6"] Jul 07 15:05:19.267682 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.267646 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvhtg\" (UniqueName: \"kubernetes.io/projected/18d9eee4-75ad-4c41-9f98-980b27ca6744-kube-api-access-lvhtg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6\" (UID: \"18d9eee4-75ad-4c41-9f98-980b27ca6744\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" Jul 07 15:05:19.267682 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.267692 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18d9eee4-75ad-4c41-9f98-980b27ca6744-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6\" (UID: \"18d9eee4-75ad-4c41-9f98-980b27ca6744\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" Jul 07 15:05:19.267905 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.267725 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18d9eee4-75ad-4c41-9f98-980b27ca6744-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6\" (UID: \"18d9eee4-75ad-4c41-9f98-980b27ca6744\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" Jul 07 15:05:19.369109 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.369071 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18d9eee4-75ad-4c41-9f98-980b27ca6744-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6\" (UID: \"18d9eee4-75ad-4c41-9f98-980b27ca6744\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" Jul 07 15:05:19.369482 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.369149 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvhtg\" (UniqueName: \"kubernetes.io/projected/18d9eee4-75ad-4c41-9f98-980b27ca6744-kube-api-access-lvhtg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6\" (UID: \"18d9eee4-75ad-4c41-9f98-980b27ca6744\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" Jul 07 15:05:19.369482 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.369173 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18d9eee4-75ad-4c41-9f98-980b27ca6744-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6\" (UID: \"18d9eee4-75ad-4c41-9f98-980b27ca6744\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" Jul 07 15:05:19.369557 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.369500 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18d9eee4-75ad-4c41-9f98-980b27ca6744-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6\" (UID: \"18d9eee4-75ad-4c41-9f98-980b27ca6744\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" Jul 07 15:05:19.369557 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.369502 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18d9eee4-75ad-4c41-9f98-980b27ca6744-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6\" (UID: \"18d9eee4-75ad-4c41-9f98-980b27ca6744\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" Jul 07 15:05:19.386426 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.386396 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvhtg\" (UniqueName: \"kubernetes.io/projected/18d9eee4-75ad-4c41-9f98-980b27ca6744-kube-api-access-lvhtg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6\" (UID: \"18d9eee4-75ad-4c41-9f98-980b27ca6744\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" Jul 07 15:05:19.485927 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.485839 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" Jul 07 15:05:19.619149 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:19.618999 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6"] Jul 07 15:05:19.621837 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:05:19.621809 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18d9eee4_75ad_4c41_9f98_980b27ca6744.slice/crio-ef44af7c8ea830fa3a160326e8076a3ae4d543d4fa1cad80f64116075dffa954 WatchSource:0}: Error finding container ef44af7c8ea830fa3a160326e8076a3ae4d543d4fa1cad80f64116075dffa954: Status 404 returned error can't find the container with id ef44af7c8ea830fa3a160326e8076a3ae4d543d4fa1cad80f64116075dffa954 Jul 07 15:05:20.298715 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:20.298680 2565 generic.go:356] "Generic (PLEG): container finished" podID="18d9eee4-75ad-4c41-9f98-980b27ca6744" containerID="a0aa7e9f458768debc9984cf585110571e5ae08f4712e292d74e39e39ba04127" exitCode=0 Jul 07 15:05:20.298917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:20.298739 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" event={"ID":"18d9eee4-75ad-4c41-9f98-980b27ca6744","Type":"ContainerDied","Data":"a0aa7e9f458768debc9984cf585110571e5ae08f4712e292d74e39e39ba04127"} Jul 07 15:05:20.298917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:20.298789 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" event={"ID":"18d9eee4-75ad-4c41-9f98-980b27ca6744","Type":"ContainerStarted","Data":"ef44af7c8ea830fa3a160326e8076a3ae4d543d4fa1cad80f64116075dffa954"} Jul 07 15:05:22.309703 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:22.309668 2565 generic.go:356] "Generic (PLEG): container finished" podID="18d9eee4-75ad-4c41-9f98-980b27ca6744" containerID="1fe3c8f978b0ba273d6ae158cc488a29defbbfd00de9c5565b9fb5641333e09f" exitCode=0 Jul 07 15:05:22.310096 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:22.309748 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" event={"ID":"18d9eee4-75ad-4c41-9f98-980b27ca6744","Type":"ContainerDied","Data":"1fe3c8f978b0ba273d6ae158cc488a29defbbfd00de9c5565b9fb5641333e09f"} Jul 07 15:05:23.243597 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.243566 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl"] Jul 07 15:05:23.245825 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.245810 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl" Jul 07 15:05:23.249189 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.249169 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 15:05:23.250053 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.250037 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 15:05:23.250125 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.250044 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-5dwcw\"" Jul 07 15:05:23.258571 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.258547 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl"] Jul 07 15:05:23.316274 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.316242 2565 generic.go:356] "Generic (PLEG): container finished" podID="18d9eee4-75ad-4c41-9f98-980b27ca6744" containerID="43987420fd32954e5ea30fcd62917609b9d636b95487be491449dfb26605e0e9" exitCode=0 Jul 07 15:05:23.316602 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.316305 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" event={"ID":"18d9eee4-75ad-4c41-9f98-980b27ca6744","Type":"ContainerDied","Data":"43987420fd32954e5ea30fcd62917609b9d636b95487be491449dfb26605e0e9"} Jul 07 15:05:23.402027 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.401992 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvpzt\" (UniqueName: \"kubernetes.io/projected/12f4000b-f366-4435-88f5-1d1f3d43008b-kube-api-access-lvpzt\") pod \"cert-manager-cainjector-559cdd8c87-cd5wl\" (UID: \"12f4000b-f366-4435-88f5-1d1f3d43008b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl" Jul 07 15:05:23.402173 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.402042 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12f4000b-f366-4435-88f5-1d1f3d43008b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cd5wl\" (UID: \"12f4000b-f366-4435-88f5-1d1f3d43008b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl" Jul 07 15:05:23.502519 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.502448 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvpzt\" (UniqueName: \"kubernetes.io/projected/12f4000b-f366-4435-88f5-1d1f3d43008b-kube-api-access-lvpzt\") pod \"cert-manager-cainjector-559cdd8c87-cd5wl\" (UID: \"12f4000b-f366-4435-88f5-1d1f3d43008b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl" Jul 07 15:05:23.502519 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.502494 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12f4000b-f366-4435-88f5-1d1f3d43008b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cd5wl\" (UID: \"12f4000b-f366-4435-88f5-1d1f3d43008b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl" Jul 07 15:05:23.511877 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.511846 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12f4000b-f366-4435-88f5-1d1f3d43008b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cd5wl\" (UID: \"12f4000b-f366-4435-88f5-1d1f3d43008b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl" Jul 07 15:05:23.511877 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.511862 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvpzt\" (UniqueName: \"kubernetes.io/projected/12f4000b-f366-4435-88f5-1d1f3d43008b-kube-api-access-lvpzt\") pod \"cert-manager-cainjector-559cdd8c87-cd5wl\" (UID: \"12f4000b-f366-4435-88f5-1d1f3d43008b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl" Jul 07 15:05:23.571605 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.571571 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl" Jul 07 15:05:23.697073 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:23.697045 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl"] Jul 07 15:05:23.699565 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:05:23.699522 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12f4000b_f366_4435_88f5_1d1f3d43008b.slice/crio-975ccdd0839bd21ef7ec4547b8cf2ed7d2daab9c3218b2542ae89d1b6dc9675c WatchSource:0}: Error finding container 975ccdd0839bd21ef7ec4547b8cf2ed7d2daab9c3218b2542ae89d1b6dc9675c: Status 404 returned error can't find the container with id 975ccdd0839bd21ef7ec4547b8cf2ed7d2daab9c3218b2542ae89d1b6dc9675c Jul 07 15:05:24.321796 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:24.321760 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl" event={"ID":"12f4000b-f366-4435-88f5-1d1f3d43008b","Type":"ContainerStarted","Data":"975ccdd0839bd21ef7ec4547b8cf2ed7d2daab9c3218b2542ae89d1b6dc9675c"} Jul 07 15:05:24.457013 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:24.456988 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" Jul 07 15:05:24.610145 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:24.610059 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvhtg\" (UniqueName: \"kubernetes.io/projected/18d9eee4-75ad-4c41-9f98-980b27ca6744-kube-api-access-lvhtg\") pod \"18d9eee4-75ad-4c41-9f98-980b27ca6744\" (UID: \"18d9eee4-75ad-4c41-9f98-980b27ca6744\") " Jul 07 15:05:24.610145 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:24.610112 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18d9eee4-75ad-4c41-9f98-980b27ca6744-util\") pod \"18d9eee4-75ad-4c41-9f98-980b27ca6744\" (UID: \"18d9eee4-75ad-4c41-9f98-980b27ca6744\") " Jul 07 15:05:24.610145 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:24.610131 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18d9eee4-75ad-4c41-9f98-980b27ca6744-bundle\") pod \"18d9eee4-75ad-4c41-9f98-980b27ca6744\" (UID: \"18d9eee4-75ad-4c41-9f98-980b27ca6744\") " Jul 07 15:05:24.610544 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:24.610519 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18d9eee4-75ad-4c41-9f98-980b27ca6744-bundle" (OuterVolumeSpecName: "bundle") pod "18d9eee4-75ad-4c41-9f98-980b27ca6744" (UID: "18d9eee4-75ad-4c41-9f98-980b27ca6744"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:05:24.612245 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:24.612221 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18d9eee4-75ad-4c41-9f98-980b27ca6744-kube-api-access-lvhtg" (OuterVolumeSpecName: "kube-api-access-lvhtg") pod "18d9eee4-75ad-4c41-9f98-980b27ca6744" (UID: "18d9eee4-75ad-4c41-9f98-980b27ca6744"). InnerVolumeSpecName "kube-api-access-lvhtg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:05:24.615587 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:24.615544 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18d9eee4-75ad-4c41-9f98-980b27ca6744-util" (OuterVolumeSpecName: "util") pod "18d9eee4-75ad-4c41-9f98-980b27ca6744" (UID: "18d9eee4-75ad-4c41-9f98-980b27ca6744"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:05:24.711619 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:24.711588 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18d9eee4-75ad-4c41-9f98-980b27ca6744-util\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:05:24.711619 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:24.711613 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18d9eee4-75ad-4c41-9f98-980b27ca6744-bundle\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:05:24.711767 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:24.711627 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvhtg\" (UniqueName: \"kubernetes.io/projected/18d9eee4-75ad-4c41-9f98-980b27ca6744-kube-api-access-lvhtg\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:05:25.328244 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:25.328205 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" event={"ID":"18d9eee4-75ad-4c41-9f98-980b27ca6744","Type":"ContainerDied","Data":"ef44af7c8ea830fa3a160326e8076a3ae4d543d4fa1cad80f64116075dffa954"} Jul 07 15:05:25.328244 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:25.328244 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxpls6" Jul 07 15:05:25.328711 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:25.328264 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef44af7c8ea830fa3a160326e8076a3ae4d543d4fa1cad80f64116075dffa954" Jul 07 15:05:26.209866 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.209824 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7lv7q"] Jul 07 15:05:26.210203 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.210182 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d9eee4-75ad-4c41-9f98-980b27ca6744" containerName="util" Jul 07 15:05:26.210203 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.210202 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d9eee4-75ad-4c41-9f98-980b27ca6744" containerName="util" Jul 07 15:05:26.210408 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.210221 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d9eee4-75ad-4c41-9f98-980b27ca6744" containerName="extract" Jul 07 15:05:26.210408 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.210230 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d9eee4-75ad-4c41-9f98-980b27ca6744" containerName="extract" Jul 07 15:05:26.210408 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.210247 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d9eee4-75ad-4c41-9f98-980b27ca6744" containerName="pull" Jul 07 15:05:26.210408 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.210256 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d9eee4-75ad-4c41-9f98-980b27ca6744" containerName="pull" Jul 07 15:05:26.210408 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.210347 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="18d9eee4-75ad-4c41-9f98-980b27ca6744" containerName="extract" Jul 07 15:05:26.212598 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.212578 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7lv7q" Jul 07 15:05:26.215050 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.215018 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-tltkm\"" Jul 07 15:05:26.223374 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.223356 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7lv7q"] Jul 07 15:05:26.325853 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.325816 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv2gc\" (UniqueName: \"kubernetes.io/projected/47b0a43e-65b9-4d61-ba8d-67882a8fc004-kube-api-access-sv2gc\") pod \"cert-manager-webhook-6486ffd997-7lv7q\" (UID: \"47b0a43e-65b9-4d61-ba8d-67882a8fc004\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7lv7q" Jul 07 15:05:26.326053 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.325883 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47b0a43e-65b9-4d61-ba8d-67882a8fc004-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7lv7q\" (UID: \"47b0a43e-65b9-4d61-ba8d-67882a8fc004\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7lv7q" Jul 07 15:05:26.426839 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.426803 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv2gc\" (UniqueName: \"kubernetes.io/projected/47b0a43e-65b9-4d61-ba8d-67882a8fc004-kube-api-access-sv2gc\") pod \"cert-manager-webhook-6486ffd997-7lv7q\" (UID: \"47b0a43e-65b9-4d61-ba8d-67882a8fc004\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7lv7q" Jul 07 15:05:26.427306 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.426862 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47b0a43e-65b9-4d61-ba8d-67882a8fc004-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7lv7q\" (UID: \"47b0a43e-65b9-4d61-ba8d-67882a8fc004\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7lv7q" Jul 07 15:05:26.437050 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.437026 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47b0a43e-65b9-4d61-ba8d-67882a8fc004-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7lv7q\" (UID: \"47b0a43e-65b9-4d61-ba8d-67882a8fc004\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7lv7q" Jul 07 15:05:26.437559 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.437533 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv2gc\" (UniqueName: \"kubernetes.io/projected/47b0a43e-65b9-4d61-ba8d-67882a8fc004-kube-api-access-sv2gc\") pod \"cert-manager-webhook-6486ffd997-7lv7q\" (UID: \"47b0a43e-65b9-4d61-ba8d-67882a8fc004\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7lv7q" Jul 07 15:05:26.522753 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.522677 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7lv7q" Jul 07 15:05:26.904002 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:26.903970 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7lv7q"] Jul 07 15:05:26.906797 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:05:26.906770 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47b0a43e_65b9_4d61_ba8d_67882a8fc004.slice/crio-45319cbf58288a51bd99654b1fe0ad1bba08eaacd9c667252397bb57ff804352 WatchSource:0}: Error finding container 45319cbf58288a51bd99654b1fe0ad1bba08eaacd9c667252397bb57ff804352: Status 404 returned error can't find the container with id 45319cbf58288a51bd99654b1fe0ad1bba08eaacd9c667252397bb57ff804352 Jul 07 15:05:27.338770 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:27.338730 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7lv7q" event={"ID":"47b0a43e-65b9-4d61-ba8d-67882a8fc004","Type":"ContainerStarted","Data":"87884e46b22ef75b9a8aaede9258f95f2b56a0dd49c514a152e3ddd41c93b733"} Jul 07 15:05:27.338770 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:27.338775 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7lv7q" event={"ID":"47b0a43e-65b9-4d61-ba8d-67882a8fc004","Type":"ContainerStarted","Data":"45319cbf58288a51bd99654b1fe0ad1bba08eaacd9c667252397bb57ff804352"} Jul 07 15:05:27.339075 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:27.338792 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7lv7q" Jul 07 15:05:27.340364 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:27.340339 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl" event={"ID":"12f4000b-f366-4435-88f5-1d1f3d43008b","Type":"ContainerStarted","Data":"a0424584ba57308d206a036571e10d66c2dc18443ebd596defa17764b4348c8e"} Jul 07 15:05:27.371786 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:27.371745 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-7lv7q" podStartSLOduration=1.371729911 podStartE2EDuration="1.371729911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:27.371612863 +0000 UTC m=+256.797001250" watchObservedRunningTime="2026-07-07 15:05:27.371729911 +0000 UTC m=+256.797118300" Jul 07 15:05:27.415461 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:27.415411 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cd5wl" podStartSLOduration=1.319176669 podStartE2EDuration="4.415393955s" totalImagesPullingTime="3.096217286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:27.413092591 +0000 UTC m=+256.838480978" watchObservedRunningTime="2026-07-07 15:05:27.415393955 +0000 UTC m=+256.840782345" Jul 07 15:05:29.346998 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:29.346960 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5h9lt"] Jul 07 15:05:29.349337 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:29.349318 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5h9lt" Jul 07 15:05:29.352153 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:29.352135 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-xn88w\"" Jul 07 15:05:29.361816 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:29.361792 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5h9lt"] Jul 07 15:05:29.459282 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:29.459242 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3e559bd-a15b-46bf-9b7c-695ea27a0f5f-bound-sa-token\") pod \"cert-manager-55b4d954d5-5h9lt\" (UID: \"a3e559bd-a15b-46bf-9b7c-695ea27a0f5f\") " pod="cert-manager/cert-manager-55b4d954d5-5h9lt" Jul 07 15:05:29.459282 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:29.459284 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96zp5\" (UniqueName: \"kubernetes.io/projected/a3e559bd-a15b-46bf-9b7c-695ea27a0f5f-kube-api-access-96zp5\") pod \"cert-manager-55b4d954d5-5h9lt\" (UID: \"a3e559bd-a15b-46bf-9b7c-695ea27a0f5f\") " pod="cert-manager/cert-manager-55b4d954d5-5h9lt" Jul 07 15:05:29.560252 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:29.560217 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3e559bd-a15b-46bf-9b7c-695ea27a0f5f-bound-sa-token\") pod \"cert-manager-55b4d954d5-5h9lt\" (UID: \"a3e559bd-a15b-46bf-9b7c-695ea27a0f5f\") " pod="cert-manager/cert-manager-55b4d954d5-5h9lt" Jul 07 15:05:29.560419 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:29.560258 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96zp5\" (UniqueName: \"kubernetes.io/projected/a3e559bd-a15b-46bf-9b7c-695ea27a0f5f-kube-api-access-96zp5\") pod \"cert-manager-55b4d954d5-5h9lt\" (UID: \"a3e559bd-a15b-46bf-9b7c-695ea27a0f5f\") " pod="cert-manager/cert-manager-55b4d954d5-5h9lt" Jul 07 15:05:29.569226 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:29.569199 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96zp5\" (UniqueName: \"kubernetes.io/projected/a3e559bd-a15b-46bf-9b7c-695ea27a0f5f-kube-api-access-96zp5\") pod \"cert-manager-55b4d954d5-5h9lt\" (UID: \"a3e559bd-a15b-46bf-9b7c-695ea27a0f5f\") " pod="cert-manager/cert-manager-55b4d954d5-5h9lt" Jul 07 15:05:29.569330 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:29.569314 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a3e559bd-a15b-46bf-9b7c-695ea27a0f5f-bound-sa-token\") pod \"cert-manager-55b4d954d5-5h9lt\" (UID: \"a3e559bd-a15b-46bf-9b7c-695ea27a0f5f\") " pod="cert-manager/cert-manager-55b4d954d5-5h9lt" Jul 07 15:05:29.658511 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:29.658420 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5h9lt" Jul 07 15:05:29.788992 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:29.788971 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5h9lt"] Jul 07 15:05:29.791075 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:05:29.791048 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3e559bd_a15b_46bf_9b7c_695ea27a0f5f.slice/crio-3527f2ce767b82e071e6a4b1c198d75f4707955593a2551fdb05b6bb766d1977 WatchSource:0}: Error finding container 3527f2ce767b82e071e6a4b1c198d75f4707955593a2551fdb05b6bb766d1977: Status 404 returned error can't find the container with id 3527f2ce767b82e071e6a4b1c198d75f4707955593a2551fdb05b6bb766d1977 Jul 07 15:05:30.355101 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:30.355064 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5h9lt" event={"ID":"a3e559bd-a15b-46bf-9b7c-695ea27a0f5f","Type":"ContainerStarted","Data":"7b9a8e025366211ee325c8737d589e61d34efc3e86802cf6bf9bb6a3a65282c9"} Jul 07 15:05:30.355495 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:30.355109 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5h9lt" event={"ID":"a3e559bd-a15b-46bf-9b7c-695ea27a0f5f","Type":"ContainerStarted","Data":"3527f2ce767b82e071e6a4b1c198d75f4707955593a2551fdb05b6bb766d1977"} Jul 07 15:05:30.380441 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:30.380395 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-5h9lt" podStartSLOduration=1.380377825 podStartE2EDuration="1.380377825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:30.377814564 +0000 UTC m=+259.803202953" watchObservedRunningTime="2026-07-07 15:05:30.380377825 +0000 UTC m=+259.805766213" Jul 07 15:05:33.347063 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:33.346974 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7lv7q" Jul 07 15:05:39.141063 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.141030 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh"] Jul 07 15:05:39.143692 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.143674 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" Jul 07 15:05:39.146825 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.146806 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 15:05:39.147854 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.147832 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 15:05:39.147968 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.147872 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-64p5b\"" Jul 07 15:05:39.160810 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.160785 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh"] Jul 07 15:05:39.241476 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.241441 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e48d6063-bde8-49a3-b2de-79fab874ec06-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh\" (UID: \"e48d6063-bde8-49a3-b2de-79fab874ec06\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" Jul 07 15:05:39.241635 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.241493 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tl2n\" (UniqueName: \"kubernetes.io/projected/e48d6063-bde8-49a3-b2de-79fab874ec06-kube-api-access-5tl2n\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh\" (UID: \"e48d6063-bde8-49a3-b2de-79fab874ec06\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" Jul 07 15:05:39.241635 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.241569 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e48d6063-bde8-49a3-b2de-79fab874ec06-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh\" (UID: \"e48d6063-bde8-49a3-b2de-79fab874ec06\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" Jul 07 15:05:39.342314 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.342278 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e48d6063-bde8-49a3-b2de-79fab874ec06-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh\" (UID: \"e48d6063-bde8-49a3-b2de-79fab874ec06\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" Jul 07 15:05:39.342476 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.342324 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tl2n\" (UniqueName: \"kubernetes.io/projected/e48d6063-bde8-49a3-b2de-79fab874ec06-kube-api-access-5tl2n\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh\" (UID: \"e48d6063-bde8-49a3-b2de-79fab874ec06\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" Jul 07 15:05:39.342476 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.342362 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e48d6063-bde8-49a3-b2de-79fab874ec06-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh\" (UID: \"e48d6063-bde8-49a3-b2de-79fab874ec06\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" Jul 07 15:05:39.342663 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.342643 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e48d6063-bde8-49a3-b2de-79fab874ec06-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh\" (UID: \"e48d6063-bde8-49a3-b2de-79fab874ec06\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" Jul 07 15:05:39.342698 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.342685 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e48d6063-bde8-49a3-b2de-79fab874ec06-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh\" (UID: \"e48d6063-bde8-49a3-b2de-79fab874ec06\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" Jul 07 15:05:39.353302 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.353275 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tl2n\" (UniqueName: \"kubernetes.io/projected/e48d6063-bde8-49a3-b2de-79fab874ec06-kube-api-access-5tl2n\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh\" (UID: \"e48d6063-bde8-49a3-b2de-79fab874ec06\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" Jul 07 15:05:39.452896 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.452871 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" Jul 07 15:05:39.592804 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:39.587799 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh"] Jul 07 15:05:40.401433 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:40.401394 2565 generic.go:356] "Generic (PLEG): container finished" podID="e48d6063-bde8-49a3-b2de-79fab874ec06" containerID="6c63b2fe2dc110203bdb79cb43d51b64ae0005beec9dd7c65621adc184f99330" exitCode=0 Jul 07 15:05:40.401875 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:40.401460 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" event={"ID":"e48d6063-bde8-49a3-b2de-79fab874ec06","Type":"ContainerDied","Data":"6c63b2fe2dc110203bdb79cb43d51b64ae0005beec9dd7c65621adc184f99330"} Jul 07 15:05:40.401875 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:40.401528 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" event={"ID":"e48d6063-bde8-49a3-b2de-79fab874ec06","Type":"ContainerStarted","Data":"06d01b12ed2aecc8e7076be28912404cfb171ffce16432e08f090b08827a9418"} Jul 07 15:05:41.408171 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:41.408097 2565 generic.go:356] "Generic (PLEG): container finished" podID="e48d6063-bde8-49a3-b2de-79fab874ec06" containerID="6847118ab5453e9e7b9db8c251fc74f6933cb53f804f8ab3871cabb4622c7558" exitCode=0 Jul 07 15:05:41.408496 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:41.408161 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" event={"ID":"e48d6063-bde8-49a3-b2de-79fab874ec06","Type":"ContainerDied","Data":"6847118ab5453e9e7b9db8c251fc74f6933cb53f804f8ab3871cabb4622c7558"} Jul 07 15:05:42.415360 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:42.415326 2565 generic.go:356] "Generic (PLEG): container finished" podID="e48d6063-bde8-49a3-b2de-79fab874ec06" containerID="30243bfd4351aaa91b016c213220719d29982c2e350a33d7684eec7cf95361b6" exitCode=0 Jul 07 15:05:42.415754 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:42.415408 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" event={"ID":"e48d6063-bde8-49a3-b2de-79fab874ec06","Type":"ContainerDied","Data":"30243bfd4351aaa91b016c213220719d29982c2e350a33d7684eec7cf95361b6"} Jul 07 15:05:43.537097 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:43.537073 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" Jul 07 15:05:43.573247 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:43.573222 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e48d6063-bde8-49a3-b2de-79fab874ec06-util\") pod \"e48d6063-bde8-49a3-b2de-79fab874ec06\" (UID: \"e48d6063-bde8-49a3-b2de-79fab874ec06\") " Jul 07 15:05:43.573389 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:43.573272 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tl2n\" (UniqueName: \"kubernetes.io/projected/e48d6063-bde8-49a3-b2de-79fab874ec06-kube-api-access-5tl2n\") pod \"e48d6063-bde8-49a3-b2de-79fab874ec06\" (UID: \"e48d6063-bde8-49a3-b2de-79fab874ec06\") " Jul 07 15:05:43.573389 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:43.573303 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e48d6063-bde8-49a3-b2de-79fab874ec06-bundle\") pod \"e48d6063-bde8-49a3-b2de-79fab874ec06\" (UID: \"e48d6063-bde8-49a3-b2de-79fab874ec06\") " Jul 07 15:05:43.574291 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:43.574258 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48d6063-bde8-49a3-b2de-79fab874ec06-bundle" (OuterVolumeSpecName: "bundle") pod "e48d6063-bde8-49a3-b2de-79fab874ec06" (UID: "e48d6063-bde8-49a3-b2de-79fab874ec06"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:05:43.575345 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:43.575320 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e48d6063-bde8-49a3-b2de-79fab874ec06-kube-api-access-5tl2n" (OuterVolumeSpecName: "kube-api-access-5tl2n") pod "e48d6063-bde8-49a3-b2de-79fab874ec06" (UID: "e48d6063-bde8-49a3-b2de-79fab874ec06"). InnerVolumeSpecName "kube-api-access-5tl2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:05:43.578953 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:43.578914 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48d6063-bde8-49a3-b2de-79fab874ec06-util" (OuterVolumeSpecName: "util") pod "e48d6063-bde8-49a3-b2de-79fab874ec06" (UID: "e48d6063-bde8-49a3-b2de-79fab874ec06"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:05:43.674560 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:43.674477 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e48d6063-bde8-49a3-b2de-79fab874ec06-util\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:05:43.674560 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:43.674504 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tl2n\" (UniqueName: \"kubernetes.io/projected/e48d6063-bde8-49a3-b2de-79fab874ec06-kube-api-access-5tl2n\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:05:43.674560 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:43.674516 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e48d6063-bde8-49a3-b2de-79fab874ec06-bundle\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:05:44.425642 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:44.425608 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" event={"ID":"e48d6063-bde8-49a3-b2de-79fab874ec06","Type":"ContainerDied","Data":"06d01b12ed2aecc8e7076be28912404cfb171ffce16432e08f090b08827a9418"} Jul 07 15:05:44.425642 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:44.425633 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdthmh" Jul 07 15:05:44.425859 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:44.425654 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06d01b12ed2aecc8e7076be28912404cfb171ffce16432e08f090b08827a9418" Jul 07 15:05:51.980134 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.980104 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth"] Jul 07 15:05:51.980615 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.980528 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e48d6063-bde8-49a3-b2de-79fab874ec06" containerName="pull" Jul 07 15:05:51.980615 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.980545 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e48d6063-bde8-49a3-b2de-79fab874ec06" containerName="pull" Jul 07 15:05:51.980615 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.980580 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e48d6063-bde8-49a3-b2de-79fab874ec06" containerName="extract" Jul 07 15:05:51.980615 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.980588 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e48d6063-bde8-49a3-b2de-79fab874ec06" containerName="extract" Jul 07 15:05:51.980615 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.980602 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e48d6063-bde8-49a3-b2de-79fab874ec06" containerName="util" Jul 07 15:05:51.980615 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.980610 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e48d6063-bde8-49a3-b2de-79fab874ec06" containerName="util" Jul 07 15:05:51.980915 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.980676 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e48d6063-bde8-49a3-b2de-79fab874ec06" containerName="extract" Jul 07 15:05:51.983238 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.983216 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:51.987262 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.987234 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 15:05:51.987395 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.987234 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 15:05:51.987395 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.987313 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8svlv\"" Jul 07 15:05:51.987395 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.987239 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:05:51.987395 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.987388 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 15:05:51.987598 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:51.987239 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 15:05:52.001612 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.001587 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth"] Jul 07 15:05:52.014280 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.014252 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8"] Jul 07 15:05:52.016731 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.016698 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.019178 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.019155 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 15:05:52.020371 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.020357 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 15:05:52.020589 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.020574 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7xnv8\"" Jul 07 15:05:52.020864 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.020852 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 15:05:52.021028 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.021009 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 15:05:52.035541 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.035520 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-tmp\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.035648 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.035549 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f67887fd-d0ea-47e4-a61e-f4208b204211-manager-config\") pod \"lws-controller-manager-d59ff9cf9-4mcth\" (UID: \"f67887fd-d0ea-47e4-a61e-f4208b204211\") " pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:52.035648 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.035568 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fk6w\" (UniqueName: \"kubernetes.io/projected/f67887fd-d0ea-47e4-a61e-f4208b204211-kube-api-access-6fk6w\") pod \"lws-controller-manager-d59ff9cf9-4mcth\" (UID: \"f67887fd-d0ea-47e4-a61e-f4208b204211\") " pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:52.035648 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.035589 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f67887fd-d0ea-47e4-a61e-f4208b204211-cert\") pod \"lws-controller-manager-d59ff9cf9-4mcth\" (UID: \"f67887fd-d0ea-47e4-a61e-f4208b204211\") " pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:52.035648 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.035642 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.035843 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.035671 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spxxd\" (UniqueName: \"kubernetes.io/projected/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-kube-api-access-spxxd\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.035843 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.035691 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f67887fd-d0ea-47e4-a61e-f4208b204211-metrics-cert\") pod \"lws-controller-manager-d59ff9cf9-4mcth\" (UID: \"f67887fd-d0ea-47e4-a61e-f4208b204211\") " pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:52.035843 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.035744 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-webhook-cert\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.035843 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.035787 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-manifests\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.058692 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.058667 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8"] Jul 07 15:05:52.136199 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.136161 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-webhook-cert\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.136199 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.136202 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-manifests\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.136418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.136250 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-tmp\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.136418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.136275 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f67887fd-d0ea-47e4-a61e-f4208b204211-manager-config\") pod \"lws-controller-manager-d59ff9cf9-4mcth\" (UID: \"f67887fd-d0ea-47e4-a61e-f4208b204211\") " pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:52.136418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.136302 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fk6w\" (UniqueName: \"kubernetes.io/projected/f67887fd-d0ea-47e4-a61e-f4208b204211-kube-api-access-6fk6w\") pod \"lws-controller-manager-d59ff9cf9-4mcth\" (UID: \"f67887fd-d0ea-47e4-a61e-f4208b204211\") " pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:52.136418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.136330 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f67887fd-d0ea-47e4-a61e-f4208b204211-cert\") pod \"lws-controller-manager-d59ff9cf9-4mcth\" (UID: \"f67887fd-d0ea-47e4-a61e-f4208b204211\") " pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:52.136418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.136356 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.136418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.136387 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spxxd\" (UniqueName: \"kubernetes.io/projected/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-kube-api-access-spxxd\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.136418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.136415 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f67887fd-d0ea-47e4-a61e-f4208b204211-metrics-cert\") pod \"lws-controller-manager-d59ff9cf9-4mcth\" (UID: \"f67887fd-d0ea-47e4-a61e-f4208b204211\") " pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:52.137119 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.137085 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-manifests\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.137244 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.137100 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f67887fd-d0ea-47e4-a61e-f4208b204211-manager-config\") pod \"lws-controller-manager-d59ff9cf9-4mcth\" (UID: \"f67887fd-d0ea-47e4-a61e-f4208b204211\") " pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:52.138649 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.138617 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-tmp\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.138909 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.138892 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.139021 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.138997 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-webhook-cert\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.139187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.139168 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f67887fd-d0ea-47e4-a61e-f4208b204211-metrics-cert\") pod \"lws-controller-manager-d59ff9cf9-4mcth\" (UID: \"f67887fd-d0ea-47e4-a61e-f4208b204211\") " pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:52.139241 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.139186 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f67887fd-d0ea-47e4-a61e-f4208b204211-cert\") pod \"lws-controller-manager-d59ff9cf9-4mcth\" (UID: \"f67887fd-d0ea-47e4-a61e-f4208b204211\") " pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:52.146902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.146876 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spxxd\" (UniqueName: \"kubernetes.io/projected/10a2db9b-1d35-4dfc-af70-cbf80dd72ff6-kube-api-access-spxxd\") pod \"opendatahub-operator-controller-manager-845dc9964f-2w2h8\" (UID: \"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.149342 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.149320 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fk6w\" (UniqueName: \"kubernetes.io/projected/f67887fd-d0ea-47e4-a61e-f4208b204211-kube-api-access-6fk6w\") pod \"lws-controller-manager-d59ff9cf9-4mcth\" (UID: \"f67887fd-d0ea-47e4-a61e-f4208b204211\") " pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:52.291997 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.291904 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:52.326841 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.326809 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:52.458092 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.457961 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth"] Jul 07 15:05:52.461672 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:05:52.461635 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf67887fd_d0ea_47e4_a61e_f4208b204211.slice/crio-662bffb6a77454afc25548ddf7f6d9389be4742f9441a0bd101f229651b81f03 WatchSource:0}: Error finding container 662bffb6a77454afc25548ddf7f6d9389be4742f9441a0bd101f229651b81f03: Status 404 returned error can't find the container with id 662bffb6a77454afc25548ddf7f6d9389be4742f9441a0bd101f229651b81f03 Jul 07 15:05:52.493170 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:52.493139 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8"] Jul 07 15:05:52.497229 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:05:52.497175 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a2db9b_1d35_4dfc_af70_cbf80dd72ff6.slice/crio-239b66383aa930df871a18f41e4856dae91ae044db56ed081b40306ce1045fec WatchSource:0}: Error finding container 239b66383aa930df871a18f41e4856dae91ae044db56ed081b40306ce1045fec: Status 404 returned error can't find the container with id 239b66383aa930df871a18f41e4856dae91ae044db56ed081b40306ce1045fec Jul 07 15:05:53.473924 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:53.473793 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" event={"ID":"f67887fd-d0ea-47e4-a61e-f4208b204211","Type":"ContainerStarted","Data":"662bffb6a77454afc25548ddf7f6d9389be4742f9441a0bd101f229651b81f03"} Jul 07 15:05:53.476222 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:53.476169 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" event={"ID":"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6","Type":"ContainerStarted","Data":"239b66383aa930df871a18f41e4856dae91ae044db56ed081b40306ce1045fec"} Jul 07 15:05:56.493739 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:56.493698 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" event={"ID":"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6","Type":"ContainerStarted","Data":"2fd0b2e182b03dedec0721583478187588560dca848cae62da118601c0bbfa62"} Jul 07 15:05:56.495229 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:56.495204 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" event={"ID":"f67887fd-d0ea-47e4-a61e-f4208b204211","Type":"ContainerStarted","Data":"f1ffe1c9721a2428f0e47157a2d4943bb942853ac166c690729d8f4dde8d8a55"} Jul 07 15:05:56.495338 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:56.495330 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:05:56.535646 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:56.535596 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" podStartSLOduration=2.067981184 podStartE2EDuration="5.535580016s" totalImagesPullingTime="3.467598832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:56.534417086 +0000 UTC m=+285.959805478" watchObservedRunningTime="2026-07-07 15:05:56.535580016 +0000 UTC m=+285.960968404" Jul 07 15:05:57.501292 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:57.501253 2565 generic.go:356] "Generic (PLEG): container finished" podID="10a2db9b-1d35-4dfc-af70-cbf80dd72ff6" containerID="2fd0b2e182b03dedec0721583478187588560dca848cae62da118601c0bbfa62" exitCode=0 Jul 07 15:05:57.501686 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:57.501343 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" event={"ID":"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6","Type":"ContainerDied","Data":"2fd0b2e182b03dedec0721583478187588560dca848cae62da118601c0bbfa62"} Jul 07 15:05:58.507294 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:58.507260 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" event={"ID":"10a2db9b-1d35-4dfc-af70-cbf80dd72ff6","Type":"ContainerStarted","Data":"249d24accffdcf9dbb43e0a2080d48af6fa8aa0382e6a6f17718130a5e57bf20"} Jul 07 15:05:58.507691 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:58.507514 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:05:58.539119 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:05:58.539071 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" podStartSLOduration=2.403150434 podStartE2EDuration="7.539055211s" totalImagesPullingTime="4.084287605s" totalInitContainerRuntime="1.051617172s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:58.535637177 +0000 UTC m=+287.961025577" watchObservedRunningTime="2026-07-07 15:05:58.539055211 +0000 UTC m=+287.964443657" Jul 07 15:06:07.504477 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:07.504447 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d59ff9cf9-4mcth" Jul 07 15:06:09.514977 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:09.514930 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-2w2h8" Jul 07 15:06:10.968323 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:10.968286 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:06:10.971962 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:10.971926 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:06:10.973913 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:10.973898 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 15:06:21.862441 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:21.862405 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z"] Jul 07 15:06:21.865782 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:21.865768 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:21.870070 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:21.870029 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 15:06:21.870972 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:21.870921 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-bzrrm\"" Jul 07 15:06:21.871102 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:21.870978 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 15:06:21.871102 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:21.870978 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 15:06:21.906297 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:21.906259 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z"] Jul 07 15:06:22.004855 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.004809 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.005106 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.004869 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dbdce28-0c70-433d-a096-683323c424ca-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.005106 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.004920 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9n2x\" (UniqueName: \"kubernetes.io/projected/0dbdce28-0c70-433d-a096-683323c424ca-kube-api-access-p9n2x\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.005106 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.005024 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.005106 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.005060 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dbdce28-0c70-433d-a096-683323c424ca-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.005106 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.005086 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dbdce28-0c70-433d-a096-683323c424ca-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.005342 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.005126 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.005342 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.005173 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.005342 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.005211 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.106243 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.106208 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dbdce28-0c70-433d-a096-683323c424ca-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.106442 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.106253 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9n2x\" (UniqueName: \"kubernetes.io/projected/0dbdce28-0c70-433d-a096-683323c424ca-kube-api-access-p9n2x\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.106442 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.106298 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.106442 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.106319 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dbdce28-0c70-433d-a096-683323c424ca-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.106442 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.106344 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dbdce28-0c70-433d-a096-683323c424ca-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.106442 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.106372 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.106442 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.106415 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.106759 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.106450 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.106759 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.106494 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.106978 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.106929 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.107125 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.107005 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.107125 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.107048 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.107125 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.107071 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.107385 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.107364 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dbdce28-0c70-433d-a096-683323c424ca-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.108757 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.108734 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dbdce28-0c70-433d-a096-683323c424ca-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.109003 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.108983 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dbdce28-0c70-433d-a096-683323c424ca-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.126829 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.126777 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dbdce28-0c70-433d-a096-683323c424ca-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.135319 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.135299 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9n2x\" (UniqueName: \"kubernetes.io/projected/0dbdce28-0c70-433d-a096-683323c424ca-kube-api-access-p9n2x\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678mnt6z\" (UID: \"0dbdce28-0c70-433d-a096-683323c424ca\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.180238 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.180205 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:22.334867 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.334826 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z"] Jul 07 15:06:22.339623 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:06:22.339597 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dbdce28_0c70_433d_a096_683323c424ca.slice/crio-5adcdfa8a1ccc012b38b0de59e6a8727829bfcc2c5246bf3d4454946d166b92d WatchSource:0}: Error finding container 5adcdfa8a1ccc012b38b0de59e6a8727829bfcc2c5246bf3d4454946d166b92d: Status 404 returned error can't find the container with id 5adcdfa8a1ccc012b38b0de59e6a8727829bfcc2c5246bf3d4454946d166b92d Jul 07 15:06:22.341675 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.341656 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:06:22.614753 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.614722 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" event={"ID":"0dbdce28-0c70-433d-a096-683323c424ca","Type":"ContainerStarted","Data":"5adcdfa8a1ccc012b38b0de59e6a8727829bfcc2c5246bf3d4454946d166b92d"} Jul 07 15:06:22.651425 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.651393 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk"] Jul 07 15:06:22.655928 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.655905 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.662298 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.662279 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 15:06:22.662418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.662312 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 15:06:22.662418 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.662398 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 15:06:22.665192 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.665174 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-f6f58\"" Jul 07 15:06:22.690654 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.690628 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk"] Jul 07 15:06:22.810931 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.810900 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.810931 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.810931 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk76x\" (UniqueName: \"kubernetes.io/projected/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-kube-api-access-xk76x\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.811200 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.811059 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.811200 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.811114 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.811200 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.811141 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.811352 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.811201 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.811352 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.811238 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.912388 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.912301 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.912388 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.912346 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.912388 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.912368 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.912908 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.912401 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.912908 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.912432 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.912908 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.912515 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.912908 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.912542 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk76x\" (UniqueName: \"kubernetes.io/projected/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-kube-api-access-xk76x\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.913174 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.913154 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.914811 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.914790 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.914918 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.914901 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.915202 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.915187 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.915247 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.915217 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.933152 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.933130 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk76x\" (UniqueName: \"kubernetes.io/projected/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-kube-api-access-xk76x\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.948037 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.948020 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1f0acb2-e34e-4c9f-af49-c2557eec47c1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5dwzk\" (UID: \"d1f0acb2-e34e-4c9f-af49-c2557eec47c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:22.965040 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:22.965014 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:23.120336 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:06:23.120302 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1f0acb2_e34e_4c9f_af49_c2557eec47c1.slice/crio-ed6df437e41ea51c2201f3da373cf37686903eb24c557d866fc15076f02575ec WatchSource:0}: Error finding container ed6df437e41ea51c2201f3da373cf37686903eb24c557d866fc15076f02575ec: Status 404 returned error can't find the container with id ed6df437e41ea51c2201f3da373cf37686903eb24c557d866fc15076f02575ec Jul 07 15:06:23.123667 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:23.123641 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk"] Jul 07 15:06:23.621840 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:23.621764 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" event={"ID":"d1f0acb2-e34e-4c9f-af49-c2557eec47c1","Type":"ContainerStarted","Data":"ed6df437e41ea51c2201f3da373cf37686903eb24c557d866fc15076f02575ec"} Jul 07 15:06:26.685177 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:26.685140 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:06:26.685544 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:26.685213 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:06:26.685544 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:26.685240 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:06:26.690162 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:26.690132 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:06:26.690248 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:26.690192 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:06:27.645987 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:27.645933 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" event={"ID":"0dbdce28-0c70-433d-a096-683323c424ca","Type":"ContainerStarted","Data":"4cde891d639f4bd5e7335a52de03af6a8d4478a33f593f37b3051fa1283f8498"} Jul 07 15:06:27.647784 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:27.647757 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" event={"ID":"d1f0acb2-e34e-4c9f-af49-c2557eec47c1","Type":"ContainerStarted","Data":"2c69ebfcd3608f6c217afbc2d515db79f229447be72affe5978eab7f67e0bb0b"} Jul 07 15:06:27.647920 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:27.647894 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:27.649693 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:27.649666 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-5dwzk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 15:06:27.649792 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:27.649731 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" podUID="d1f0acb2-e34e-4c9f-af49-c2557eec47c1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 15:06:27.737491 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:27.737427 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" podStartSLOduration=2.394353665 podStartE2EDuration="6.73740283s" totalImagesPullingTime="4.343049165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:06:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:06:27.708538848 +0000 UTC m=+317.133927239" watchObservedRunningTime="2026-07-07 15:06:27.73740283 +0000 UTC m=+317.162791218" Jul 07 15:06:27.737970 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:27.737584 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" podStartSLOduration=2.170712974 podStartE2EDuration="5.737578961s" totalImagesPullingTime="3.566865987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:06:27.735772875 +0000 UTC m=+317.161161292" watchObservedRunningTime="2026-07-07 15:06:27.737578961 +0000 UTC m=+317.162967367" Jul 07 15:06:28.180817 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:28.180782 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:28.185428 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:28.185404 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:28.654390 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:28.654286 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:06:28.654793 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:28.654776 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5dwzk" Jul 07 15:06:28.655328 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:06:28.655313 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678mnt6z" Jul 07 15:07:25.809992 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:25.809956 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb"] Jul 07 15:07:25.812483 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:25.812462 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" Jul 07 15:07:25.815757 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:25.815740 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-29wfl\"" Jul 07 15:07:25.815919 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:25.815908 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 15:07:25.816734 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:25.816717 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 15:07:25.823668 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:25.823647 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb"] Jul 07 15:07:25.967540 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:25.967506 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96rm2\" (UniqueName: \"kubernetes.io/projected/8a99e0df-82c6-4f71-b61c-41c8069311d2-kube-api-access-96rm2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb\" (UID: \"8a99e0df-82c6-4f71-b61c-41c8069311d2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" Jul 07 15:07:25.967736 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:25.967559 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a99e0df-82c6-4f71-b61c-41c8069311d2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb\" (UID: \"8a99e0df-82c6-4f71-b61c-41c8069311d2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" Jul 07 15:07:25.967736 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:25.967583 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a99e0df-82c6-4f71-b61c-41c8069311d2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb\" (UID: \"8a99e0df-82c6-4f71-b61c-41c8069311d2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" Jul 07 15:07:26.068964 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.068841 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a99e0df-82c6-4f71-b61c-41c8069311d2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb\" (UID: \"8a99e0df-82c6-4f71-b61c-41c8069311d2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" Jul 07 15:07:26.068964 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.068907 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a99e0df-82c6-4f71-b61c-41c8069311d2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb\" (UID: \"8a99e0df-82c6-4f71-b61c-41c8069311d2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" Jul 07 15:07:26.069186 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.069038 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96rm2\" (UniqueName: \"kubernetes.io/projected/8a99e0df-82c6-4f71-b61c-41c8069311d2-kube-api-access-96rm2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb\" (UID: \"8a99e0df-82c6-4f71-b61c-41c8069311d2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" Jul 07 15:07:26.069266 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.069243 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a99e0df-82c6-4f71-b61c-41c8069311d2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb\" (UID: \"8a99e0df-82c6-4f71-b61c-41c8069311d2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" Jul 07 15:07:26.069319 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.069249 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a99e0df-82c6-4f71-b61c-41c8069311d2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb\" (UID: \"8a99e0df-82c6-4f71-b61c-41c8069311d2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" Jul 07 15:07:26.084461 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.084435 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96rm2\" (UniqueName: \"kubernetes.io/projected/8a99e0df-82c6-4f71-b61c-41c8069311d2-kube-api-access-96rm2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb\" (UID: \"8a99e0df-82c6-4f71-b61c-41c8069311d2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" Jul 07 15:07:26.121304 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.121270 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" Jul 07 15:07:26.251740 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.251697 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb"] Jul 07 15:07:26.254140 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:07:26.254113 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a99e0df_82c6_4f71_b61c_41c8069311d2.slice/crio-49512c149b858f4ae045c1c877b03a9bb0f412fd315e8a1c422e9086ec5e1040 WatchSource:0}: Error finding container 49512c149b858f4ae045c1c877b03a9bb0f412fd315e8a1c422e9086ec5e1040: Status 404 returned error can't find the container with id 49512c149b858f4ae045c1c877b03a9bb0f412fd315e8a1c422e9086ec5e1040 Jul 07 15:07:26.406127 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.406097 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9"] Jul 07 15:07:26.408508 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.408489 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" Jul 07 15:07:26.419599 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.419576 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9"] Jul 07 15:07:26.574294 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.574259 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7wxz\" (UniqueName: \"kubernetes.io/projected/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-kube-api-access-d7wxz\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9\" (UID: \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" Jul 07 15:07:26.574461 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.574302 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9\" (UID: \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" Jul 07 15:07:26.574461 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.574322 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9\" (UID: \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" Jul 07 15:07:26.675620 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.675593 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7wxz\" (UniqueName: \"kubernetes.io/projected/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-kube-api-access-d7wxz\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9\" (UID: \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" Jul 07 15:07:26.675757 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.675630 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9\" (UID: \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" Jul 07 15:07:26.675757 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.675651 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9\" (UID: \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" Jul 07 15:07:26.676092 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.676073 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9\" (UID: \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" Jul 07 15:07:26.676136 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.676090 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9\" (UID: \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" Jul 07 15:07:26.686385 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.686367 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7wxz\" (UniqueName: \"kubernetes.io/projected/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-kube-api-access-d7wxz\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9\" (UID: \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" Jul 07 15:07:26.717753 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.717732 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" Jul 07 15:07:26.845440 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.845412 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9"] Jul 07 15:07:26.847177 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:07:26.847150 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1db2e0e_d5ae_4949_83dd_8e2b07d50247.slice/crio-0c66ea725cf12955e3ccbb04ce5c0b17e4a30223b34d5b53ef22472e07891b7b WatchSource:0}: Error finding container 0c66ea725cf12955e3ccbb04ce5c0b17e4a30223b34d5b53ef22472e07891b7b: Status 404 returned error can't find the container with id 0c66ea725cf12955e3ccbb04ce5c0b17e4a30223b34d5b53ef22472e07891b7b Jul 07 15:07:26.917922 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.917887 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" event={"ID":"e1db2e0e-d5ae-4949-83dd-8e2b07d50247","Type":"ContainerStarted","Data":"e19c9662880016cd6b69ab09067261792c0aceaba44a9197597c3c849dbc7d9c"} Jul 07 15:07:26.918067 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.917991 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" event={"ID":"e1db2e0e-d5ae-4949-83dd-8e2b07d50247","Type":"ContainerStarted","Data":"0c66ea725cf12955e3ccbb04ce5c0b17e4a30223b34d5b53ef22472e07891b7b"} Jul 07 15:07:26.919675 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.919653 2565 generic.go:356] "Generic (PLEG): container finished" podID="8a99e0df-82c6-4f71-b61c-41c8069311d2" containerID="708a0737379a75481efcb80642c4872590fd8ade57c6339bfcb6c5e2a7cb087f" exitCode=0 Jul 07 15:07:26.919770 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.919692 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" event={"ID":"8a99e0df-82c6-4f71-b61c-41c8069311d2","Type":"ContainerDied","Data":"708a0737379a75481efcb80642c4872590fd8ade57c6339bfcb6c5e2a7cb087f"} Jul 07 15:07:26.919770 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.919724 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" event={"ID":"8a99e0df-82c6-4f71-b61c-41c8069311d2","Type":"ContainerStarted","Data":"49512c149b858f4ae045c1c877b03a9bb0f412fd315e8a1c422e9086ec5e1040"} Jul 07 15:07:26.997581 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.997551 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh"] Jul 07 15:07:26.999883 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:26.999867 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" Jul 07 15:07:27.012839 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.012810 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh"] Jul 07 15:07:27.181141 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.181104 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzd6s\" (UniqueName: \"kubernetes.io/projected/fb5faf61-8542-4065-a260-31669e2af05f-kube-api-access-mzd6s\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh\" (UID: \"fb5faf61-8542-4065-a260-31669e2af05f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" Jul 07 15:07:27.181308 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.181184 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb5faf61-8542-4065-a260-31669e2af05f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh\" (UID: \"fb5faf61-8542-4065-a260-31669e2af05f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" Jul 07 15:07:27.181308 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.181248 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb5faf61-8542-4065-a260-31669e2af05f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh\" (UID: \"fb5faf61-8542-4065-a260-31669e2af05f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" Jul 07 15:07:27.281986 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.281901 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb5faf61-8542-4065-a260-31669e2af05f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh\" (UID: \"fb5faf61-8542-4065-a260-31669e2af05f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" Jul 07 15:07:27.282111 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.281981 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb5faf61-8542-4065-a260-31669e2af05f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh\" (UID: \"fb5faf61-8542-4065-a260-31669e2af05f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" Jul 07 15:07:27.282111 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.282041 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzd6s\" (UniqueName: \"kubernetes.io/projected/fb5faf61-8542-4065-a260-31669e2af05f-kube-api-access-mzd6s\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh\" (UID: \"fb5faf61-8542-4065-a260-31669e2af05f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" Jul 07 15:07:27.282361 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.282342 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb5faf61-8542-4065-a260-31669e2af05f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh\" (UID: \"fb5faf61-8542-4065-a260-31669e2af05f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" Jul 07 15:07:27.282401 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.282360 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb5faf61-8542-4065-a260-31669e2af05f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh\" (UID: \"fb5faf61-8542-4065-a260-31669e2af05f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" Jul 07 15:07:27.292509 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.292488 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzd6s\" (UniqueName: \"kubernetes.io/projected/fb5faf61-8542-4065-a260-31669e2af05f-kube-api-access-mzd6s\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh\" (UID: \"fb5faf61-8542-4065-a260-31669e2af05f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" Jul 07 15:07:27.308327 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.308302 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" Jul 07 15:07:27.440298 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.440273 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh"] Jul 07 15:07:27.442119 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:07:27.442089 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb5faf61_8542_4065_a260_31669e2af05f.slice/crio-8c1cca5118a08f4d7f15abcea66d739874f707fe4bfbfe363974dc8a518f544e WatchSource:0}: Error finding container 8c1cca5118a08f4d7f15abcea66d739874f707fe4bfbfe363974dc8a518f544e: Status 404 returned error can't find the container with id 8c1cca5118a08f4d7f15abcea66d739874f707fe4bfbfe363974dc8a518f544e Jul 07 15:07:27.598481 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.598448 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw"] Jul 07 15:07:27.600910 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.600894 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" Jul 07 15:07:27.615688 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.615666 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw"] Jul 07 15:07:27.686266 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.686238 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw\" (UID: \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" Jul 07 15:07:27.686429 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.686289 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7zsq\" (UniqueName: \"kubernetes.io/projected/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-kube-api-access-g7zsq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw\" (UID: \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" Jul 07 15:07:27.686429 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.686322 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw\" (UID: \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" Jul 07 15:07:27.787616 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.787543 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw\" (UID: \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" Jul 07 15:07:27.787616 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.787591 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7zsq\" (UniqueName: \"kubernetes.io/projected/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-kube-api-access-g7zsq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw\" (UID: \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" Jul 07 15:07:27.787616 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.787611 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw\" (UID: \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" Jul 07 15:07:27.787918 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.787902 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw\" (UID: \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" Jul 07 15:07:27.788020 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.787997 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw\" (UID: \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" Jul 07 15:07:27.797641 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.797619 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7zsq\" (UniqueName: \"kubernetes.io/projected/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-kube-api-access-g7zsq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw\" (UID: \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" Jul 07 15:07:27.909518 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.909499 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" Jul 07 15:07:27.926798 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.926772 2565 generic.go:356] "Generic (PLEG): container finished" podID="e1db2e0e-d5ae-4949-83dd-8e2b07d50247" containerID="e19c9662880016cd6b69ab09067261792c0aceaba44a9197597c3c849dbc7d9c" exitCode=0 Jul 07 15:07:27.926913 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.926857 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" event={"ID":"e1db2e0e-d5ae-4949-83dd-8e2b07d50247","Type":"ContainerDied","Data":"e19c9662880016cd6b69ab09067261792c0aceaba44a9197597c3c849dbc7d9c"} Jul 07 15:07:27.928663 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.928633 2565 generic.go:356] "Generic (PLEG): container finished" podID="fb5faf61-8542-4065-a260-31669e2af05f" containerID="1836b5ae5ff8a2c8f1fc9b34dcedf771712ac785f6de144547640ec0d518cb51" exitCode=0 Jul 07 15:07:27.928765 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.928666 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" event={"ID":"fb5faf61-8542-4065-a260-31669e2af05f","Type":"ContainerDied","Data":"1836b5ae5ff8a2c8f1fc9b34dcedf771712ac785f6de144547640ec0d518cb51"} Jul 07 15:07:27.928765 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.928717 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" event={"ID":"fb5faf61-8542-4065-a260-31669e2af05f","Type":"ContainerStarted","Data":"8c1cca5118a08f4d7f15abcea66d739874f707fe4bfbfe363974dc8a518f544e"} Jul 07 15:07:27.931237 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:27.931191 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" event={"ID":"8a99e0df-82c6-4f71-b61c-41c8069311d2","Type":"ContainerStarted","Data":"5540d3485f3b46292b75abe07ef9dbd7fe6310571d75500a1c35db27c348bd2f"} Jul 07 15:07:28.045047 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:28.045022 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw"] Jul 07 15:07:28.046589 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:07:28.046568 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5cf3c2a_1a83_407e_bcad_a03da4e78d1c.slice/crio-d980b96aa1eb9e4991b832d3e12f7469addc23395fae49c7bdb9c4a395fb71ce WatchSource:0}: Error finding container d980b96aa1eb9e4991b832d3e12f7469addc23395fae49c7bdb9c4a395fb71ce: Status 404 returned error can't find the container with id d980b96aa1eb9e4991b832d3e12f7469addc23395fae49c7bdb9c4a395fb71ce Jul 07 15:07:28.939132 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:28.939096 2565 generic.go:356] "Generic (PLEG): container finished" podID="8a99e0df-82c6-4f71-b61c-41c8069311d2" containerID="5540d3485f3b46292b75abe07ef9dbd7fe6310571d75500a1c35db27c348bd2f" exitCode=0 Jul 07 15:07:28.939468 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:28.939188 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" event={"ID":"8a99e0df-82c6-4f71-b61c-41c8069311d2","Type":"ContainerDied","Data":"5540d3485f3b46292b75abe07ef9dbd7fe6310571d75500a1c35db27c348bd2f"} Jul 07 15:07:28.941274 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:28.941251 2565 generic.go:356] "Generic (PLEG): container finished" podID="fb5faf61-8542-4065-a260-31669e2af05f" containerID="c61d73fe9ea008ff967631ac2545dec1d77ef50889a7d08f942a5d7f9de668a9" exitCode=0 Jul 07 15:07:28.941375 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:28.941326 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" event={"ID":"fb5faf61-8542-4065-a260-31669e2af05f","Type":"ContainerDied","Data":"c61d73fe9ea008ff967631ac2545dec1d77ef50889a7d08f942a5d7f9de668a9"} Jul 07 15:07:28.943532 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:28.943510 2565 generic.go:356] "Generic (PLEG): container finished" podID="f5cf3c2a-1a83-407e-bcad-a03da4e78d1c" containerID="7923875dac2aa57ff0c95475c8ce474c02dae71a8b80912db6d9ba0de9366c42" exitCode=0 Jul 07 15:07:28.943609 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:28.943587 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" event={"ID":"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c","Type":"ContainerDied","Data":"7923875dac2aa57ff0c95475c8ce474c02dae71a8b80912db6d9ba0de9366c42"} Jul 07 15:07:28.943650 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:28.943628 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" event={"ID":"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c","Type":"ContainerStarted","Data":"d980b96aa1eb9e4991b832d3e12f7469addc23395fae49c7bdb9c4a395fb71ce"} Jul 07 15:07:29.950926 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:29.950888 2565 generic.go:356] "Generic (PLEG): container finished" podID="fb5faf61-8542-4065-a260-31669e2af05f" containerID="6cfe9b258e3be724e0bf6b0b01d3797e3b6b8ac59be5af0b44c65b69c677e8b6" exitCode=0 Jul 07 15:07:29.951357 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:29.950989 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" event={"ID":"fb5faf61-8542-4065-a260-31669e2af05f","Type":"ContainerDied","Data":"6cfe9b258e3be724e0bf6b0b01d3797e3b6b8ac59be5af0b44c65b69c677e8b6"} Jul 07 15:07:29.952869 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:29.952846 2565 generic.go:356] "Generic (PLEG): container finished" podID="f5cf3c2a-1a83-407e-bcad-a03da4e78d1c" containerID="f1fc9cf717ee018cb9f18fa5da148d1118881f7731a8debe915bf1153918970e" exitCode=0 Jul 07 15:07:29.952976 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:29.952932 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" event={"ID":"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c","Type":"ContainerDied","Data":"f1fc9cf717ee018cb9f18fa5da148d1118881f7731a8debe915bf1153918970e"} Jul 07 15:07:29.955024 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:29.955002 2565 generic.go:356] "Generic (PLEG): container finished" podID="8a99e0df-82c6-4f71-b61c-41c8069311d2" containerID="5e88d87bacad37dbb24b2af8182eb41752a2d68b9b411624767bfadffcf52fa8" exitCode=0 Jul 07 15:07:29.955113 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:29.955088 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" event={"ID":"8a99e0df-82c6-4f71-b61c-41c8069311d2","Type":"ContainerDied","Data":"5e88d87bacad37dbb24b2af8182eb41752a2d68b9b411624767bfadffcf52fa8"} Jul 07 15:07:29.956833 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:29.956813 2565 generic.go:356] "Generic (PLEG): container finished" podID="e1db2e0e-d5ae-4949-83dd-8e2b07d50247" containerID="2035770b3a51dc06840b06a8568ec65f9ab7388074cd630b93bd5959e2df4722" exitCode=0 Jul 07 15:07:29.956987 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:29.956844 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" event={"ID":"e1db2e0e-d5ae-4949-83dd-8e2b07d50247","Type":"ContainerDied","Data":"2035770b3a51dc06840b06a8568ec65f9ab7388074cd630b93bd5959e2df4722"} Jul 07 15:07:30.964423 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:30.964389 2565 generic.go:356] "Generic (PLEG): container finished" podID="e1db2e0e-d5ae-4949-83dd-8e2b07d50247" containerID="44c000eb46c17dceac44e860f24b583a5b751b56274a6a53bd9efe8accf65c1e" exitCode=0 Jul 07 15:07:30.964817 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:30.964472 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" event={"ID":"e1db2e0e-d5ae-4949-83dd-8e2b07d50247","Type":"ContainerDied","Data":"44c000eb46c17dceac44e860f24b583a5b751b56274a6a53bd9efe8accf65c1e"} Jul 07 15:07:30.966490 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:30.966468 2565 generic.go:356] "Generic (PLEG): container finished" podID="f5cf3c2a-1a83-407e-bcad-a03da4e78d1c" containerID="e5791f074baa5e8ffef775a9f3cc8eddb89dca2c794f70e299897f2ba3a481dc" exitCode=0 Jul 07 15:07:30.966604 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:30.966538 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" event={"ID":"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c","Type":"ContainerDied","Data":"e5791f074baa5e8ffef775a9f3cc8eddb89dca2c794f70e299897f2ba3a481dc"} Jul 07 15:07:31.123955 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.123916 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" Jul 07 15:07:31.127022 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.127000 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" Jul 07 15:07:31.213107 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.213071 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb5faf61-8542-4065-a260-31669e2af05f-util\") pod \"fb5faf61-8542-4065-a260-31669e2af05f\" (UID: \"fb5faf61-8542-4065-a260-31669e2af05f\") " Jul 07 15:07:31.213107 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.213109 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a99e0df-82c6-4f71-b61c-41c8069311d2-bundle\") pod \"8a99e0df-82c6-4f71-b61c-41c8069311d2\" (UID: \"8a99e0df-82c6-4f71-b61c-41c8069311d2\") " Jul 07 15:07:31.213333 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.213137 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzd6s\" (UniqueName: \"kubernetes.io/projected/fb5faf61-8542-4065-a260-31669e2af05f-kube-api-access-mzd6s\") pod \"fb5faf61-8542-4065-a260-31669e2af05f\" (UID: \"fb5faf61-8542-4065-a260-31669e2af05f\") " Jul 07 15:07:31.213333 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.213167 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb5faf61-8542-4065-a260-31669e2af05f-bundle\") pod \"fb5faf61-8542-4065-a260-31669e2af05f\" (UID: \"fb5faf61-8542-4065-a260-31669e2af05f\") " Jul 07 15:07:31.213333 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.213214 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a99e0df-82c6-4f71-b61c-41c8069311d2-util\") pod \"8a99e0df-82c6-4f71-b61c-41c8069311d2\" (UID: \"8a99e0df-82c6-4f71-b61c-41c8069311d2\") " Jul 07 15:07:31.213333 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.213240 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96rm2\" (UniqueName: \"kubernetes.io/projected/8a99e0df-82c6-4f71-b61c-41c8069311d2-kube-api-access-96rm2\") pod \"8a99e0df-82c6-4f71-b61c-41c8069311d2\" (UID: \"8a99e0df-82c6-4f71-b61c-41c8069311d2\") " Jul 07 15:07:31.213791 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.213759 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb5faf61-8542-4065-a260-31669e2af05f-bundle" (OuterVolumeSpecName: "bundle") pod "fb5faf61-8542-4065-a260-31669e2af05f" (UID: "fb5faf61-8542-4065-a260-31669e2af05f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:31.214018 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.213885 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a99e0df-82c6-4f71-b61c-41c8069311d2-bundle" (OuterVolumeSpecName: "bundle") pod "8a99e0df-82c6-4f71-b61c-41c8069311d2" (UID: "8a99e0df-82c6-4f71-b61c-41c8069311d2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:31.215562 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.215493 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb5faf61-8542-4065-a260-31669e2af05f-kube-api-access-mzd6s" (OuterVolumeSpecName: "kube-api-access-mzd6s") pod "fb5faf61-8542-4065-a260-31669e2af05f" (UID: "fb5faf61-8542-4065-a260-31669e2af05f"). InnerVolumeSpecName "kube-api-access-mzd6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:07:31.215562 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.215516 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a99e0df-82c6-4f71-b61c-41c8069311d2-kube-api-access-96rm2" (OuterVolumeSpecName: "kube-api-access-96rm2") pod "8a99e0df-82c6-4f71-b61c-41c8069311d2" (UID: "8a99e0df-82c6-4f71-b61c-41c8069311d2"). InnerVolumeSpecName "kube-api-access-96rm2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:07:31.219114 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.219089 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb5faf61-8542-4065-a260-31669e2af05f-util" (OuterVolumeSpecName: "util") pod "fb5faf61-8542-4065-a260-31669e2af05f" (UID: "fb5faf61-8542-4065-a260-31669e2af05f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:31.219343 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.219319 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a99e0df-82c6-4f71-b61c-41c8069311d2-util" (OuterVolumeSpecName: "util") pod "8a99e0df-82c6-4f71-b61c-41c8069311d2" (UID: "8a99e0df-82c6-4f71-b61c-41c8069311d2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:31.314098 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.314057 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb5faf61-8542-4065-a260-31669e2af05f-bundle\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:07:31.314098 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.314089 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a99e0df-82c6-4f71-b61c-41c8069311d2-util\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:07:31.314098 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.314100 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96rm2\" (UniqueName: \"kubernetes.io/projected/8a99e0df-82c6-4f71-b61c-41c8069311d2-kube-api-access-96rm2\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:07:31.314098 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.314110 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb5faf61-8542-4065-a260-31669e2af05f-util\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:07:31.314358 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.314118 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a99e0df-82c6-4f71-b61c-41c8069311d2-bundle\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:07:31.314358 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.314126 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzd6s\" (UniqueName: \"kubernetes.io/projected/fb5faf61-8542-4065-a260-31669e2af05f-kube-api-access-mzd6s\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:07:31.973345 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.973319 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" Jul 07 15:07:31.973737 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.973345 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh" event={"ID":"fb5faf61-8542-4065-a260-31669e2af05f","Type":"ContainerDied","Data":"8c1cca5118a08f4d7f15abcea66d739874f707fe4bfbfe363974dc8a518f544e"} Jul 07 15:07:31.973737 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.973427 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c1cca5118a08f4d7f15abcea66d739874f707fe4bfbfe363974dc8a518f544e" Jul 07 15:07:31.975289 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.975272 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" Jul 07 15:07:31.975411 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.975300 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb" event={"ID":"8a99e0df-82c6-4f71-b61c-41c8069311d2","Type":"ContainerDied","Data":"49512c149b858f4ae045c1c877b03a9bb0f412fd315e8a1c422e9086ec5e1040"} Jul 07 15:07:31.975411 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:31.975341 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49512c149b858f4ae045c1c877b03a9bb0f412fd315e8a1c422e9086ec5e1040" Jul 07 15:07:32.109877 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.109851 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" Jul 07 15:07:32.120707 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.120686 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7zsq\" (UniqueName: \"kubernetes.io/projected/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-kube-api-access-g7zsq\") pod \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\" (UID: \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\") " Jul 07 15:07:32.120798 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.120725 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-util\") pod \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\" (UID: \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\") " Jul 07 15:07:32.120798 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.120774 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-bundle\") pod \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\" (UID: \"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c\") " Jul 07 15:07:32.121522 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.121496 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-bundle" (OuterVolumeSpecName: "bundle") pod "f5cf3c2a-1a83-407e-bcad-a03da4e78d1c" (UID: "f5cf3c2a-1a83-407e-bcad-a03da4e78d1c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:32.122853 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.122829 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-kube-api-access-g7zsq" (OuterVolumeSpecName: "kube-api-access-g7zsq") pod "f5cf3c2a-1a83-407e-bcad-a03da4e78d1c" (UID: "f5cf3c2a-1a83-407e-bcad-a03da4e78d1c"). InnerVolumeSpecName "kube-api-access-g7zsq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:07:32.128749 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.128729 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" Jul 07 15:07:32.129395 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.129374 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-util" (OuterVolumeSpecName: "util") pod "f5cf3c2a-1a83-407e-bcad-a03da4e78d1c" (UID: "f5cf3c2a-1a83-407e-bcad-a03da4e78d1c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:32.221515 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.221491 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7wxz\" (UniqueName: \"kubernetes.io/projected/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-kube-api-access-d7wxz\") pod \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\" (UID: \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\") " Jul 07 15:07:32.221613 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.221529 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-util\") pod \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\" (UID: \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\") " Jul 07 15:07:32.221656 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.221625 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-bundle\") pod \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\" (UID: \"e1db2e0e-d5ae-4949-83dd-8e2b07d50247\") " Jul 07 15:07:32.221848 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.221834 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-bundle\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:07:32.221955 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.221861 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7zsq\" (UniqueName: \"kubernetes.io/projected/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-kube-api-access-g7zsq\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:07:32.221955 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.221872 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5cf3c2a-1a83-407e-bcad-a03da4e78d1c-util\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:07:32.222200 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.222173 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-bundle" (OuterVolumeSpecName: "bundle") pod "e1db2e0e-d5ae-4949-83dd-8e2b07d50247" (UID: "e1db2e0e-d5ae-4949-83dd-8e2b07d50247"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:32.223608 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.223550 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-kube-api-access-d7wxz" (OuterVolumeSpecName: "kube-api-access-d7wxz") pod "e1db2e0e-d5ae-4949-83dd-8e2b07d50247" (UID: "e1db2e0e-d5ae-4949-83dd-8e2b07d50247"). InnerVolumeSpecName "kube-api-access-d7wxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:07:32.228911 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.228887 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-util" (OuterVolumeSpecName: "util") pod "e1db2e0e-d5ae-4949-83dd-8e2b07d50247" (UID: "e1db2e0e-d5ae-4949-83dd-8e2b07d50247"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:07:32.322490 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.322460 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7wxz\" (UniqueName: \"kubernetes.io/projected/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-kube-api-access-d7wxz\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:07:32.322490 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.322483 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-util\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:07:32.322490 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.322496 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1db2e0e-d5ae-4949-83dd-8e2b07d50247-bundle\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:07:32.981813 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.981786 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" Jul 07 15:07:32.982304 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.981815 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw" event={"ID":"f5cf3c2a-1a83-407e-bcad-a03da4e78d1c","Type":"ContainerDied","Data":"d980b96aa1eb9e4991b832d3e12f7469addc23395fae49c7bdb9c4a395fb71ce"} Jul 07 15:07:32.982304 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.981870 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d980b96aa1eb9e4991b832d3e12f7469addc23395fae49c7bdb9c4a395fb71ce" Jul 07 15:07:32.983547 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.983522 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" event={"ID":"e1db2e0e-d5ae-4949-83dd-8e2b07d50247","Type":"ContainerDied","Data":"0c66ea725cf12955e3ccbb04ce5c0b17e4a30223b34d5b53ef22472e07891b7b"} Jul 07 15:07:32.983658 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.983564 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c66ea725cf12955e3ccbb04ce5c0b17e4a30223b34d5b53ef22472e07891b7b" Jul 07 15:07:32.983658 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:07:32.983570 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9" Jul 07 15:08:14.813298 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813264 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjt6x"] Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813578 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5cf3c2a-1a83-407e-bcad-a03da4e78d1c" containerName="pull" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813589 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5cf3c2a-1a83-407e-bcad-a03da4e78d1c" containerName="pull" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813598 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb5faf61-8542-4065-a260-31669e2af05f" containerName="util" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813603 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb5faf61-8542-4065-a260-31669e2af05f" containerName="util" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813621 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb5faf61-8542-4065-a260-31669e2af05f" containerName="pull" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813629 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb5faf61-8542-4065-a260-31669e2af05f" containerName="pull" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813638 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1db2e0e-d5ae-4949-83dd-8e2b07d50247" containerName="pull" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813645 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1db2e0e-d5ae-4949-83dd-8e2b07d50247" containerName="pull" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813651 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb5faf61-8542-4065-a260-31669e2af05f" containerName="extract" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813659 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb5faf61-8542-4065-a260-31669e2af05f" containerName="extract" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813668 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5cf3c2a-1a83-407e-bcad-a03da4e78d1c" containerName="extract" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813676 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5cf3c2a-1a83-407e-bcad-a03da4e78d1c" containerName="extract" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813687 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5cf3c2a-1a83-407e-bcad-a03da4e78d1c" containerName="util" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813695 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5cf3c2a-1a83-407e-bcad-a03da4e78d1c" containerName="util" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813714 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a99e0df-82c6-4f71-b61c-41c8069311d2" containerName="util" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813721 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a99e0df-82c6-4f71-b61c-41c8069311d2" containerName="util" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813732 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1db2e0e-d5ae-4949-83dd-8e2b07d50247" containerName="extract" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813741 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1db2e0e-d5ae-4949-83dd-8e2b07d50247" containerName="extract" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813750 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1db2e0e-d5ae-4949-83dd-8e2b07d50247" containerName="util" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813757 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1db2e0e-d5ae-4949-83dd-8e2b07d50247" containerName="util" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813770 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a99e0df-82c6-4f71-b61c-41c8069311d2" containerName="extract" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813778 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a99e0df-82c6-4f71-b61c-41c8069311d2" containerName="extract" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813792 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a99e0df-82c6-4f71-b61c-41c8069311d2" containerName="pull" Jul 07 15:08:14.813856 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813799 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a99e0df-82c6-4f71-b61c-41c8069311d2" containerName="pull" Jul 07 15:08:14.814576 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813878 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5cf3c2a-1a83-407e-bcad-a03da4e78d1c" containerName="extract" Jul 07 15:08:14.814576 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813890 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a99e0df-82c6-4f71-b61c-41c8069311d2" containerName="extract" Jul 07 15:08:14.814576 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813901 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb5faf61-8542-4065-a260-31669e2af05f" containerName="extract" Jul 07 15:08:14.814576 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.813909 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1db2e0e-d5ae-4949-83dd-8e2b07d50247" containerName="extract" Jul 07 15:08:14.816992 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.816969 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjt6x" Jul 07 15:08:14.820092 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.820066 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 15:08:14.820220 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.820076 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 15:08:14.820930 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.820906 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-92nzr\"" Jul 07 15:08:14.830299 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.830279 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjt6x"] Jul 07 15:08:14.978429 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:14.978397 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9bm4\" (UniqueName: \"kubernetes.io/projected/6e56a8b2-e0b9-4f3e-a5df-bb0688ae15fd-kube-api-access-x9bm4\") pod \"limitador-operator-controller-manager-859d7b78bd-cjt6x\" (UID: \"6e56a8b2-e0b9-4f3e-a5df-bb0688ae15fd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjt6x" Jul 07 15:08:15.079559 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:15.079482 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9bm4\" (UniqueName: \"kubernetes.io/projected/6e56a8b2-e0b9-4f3e-a5df-bb0688ae15fd-kube-api-access-x9bm4\") pod \"limitador-operator-controller-manager-859d7b78bd-cjt6x\" (UID: \"6e56a8b2-e0b9-4f3e-a5df-bb0688ae15fd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjt6x" Jul 07 15:08:15.092972 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:15.092912 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9bm4\" (UniqueName: \"kubernetes.io/projected/6e56a8b2-e0b9-4f3e-a5df-bb0688ae15fd-kube-api-access-x9bm4\") pod \"limitador-operator-controller-manager-859d7b78bd-cjt6x\" (UID: \"6e56a8b2-e0b9-4f3e-a5df-bb0688ae15fd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjt6x" Jul 07 15:08:15.128977 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:15.128952 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjt6x" Jul 07 15:08:15.266674 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:15.266648 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjt6x"] Jul 07 15:08:15.269242 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:08:15.269206 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e56a8b2_e0b9_4f3e_a5df_bb0688ae15fd.slice/crio-f2bf0f699b98633d511a2760b163e208920d6d3f785337d26db15c251db5ca33 WatchSource:0}: Error finding container f2bf0f699b98633d511a2760b163e208920d6d3f785337d26db15c251db5ca33: Status 404 returned error can't find the container with id f2bf0f699b98633d511a2760b163e208920d6d3f785337d26db15c251db5ca33 Jul 07 15:08:16.202409 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:16.202372 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjt6x" event={"ID":"6e56a8b2-e0b9-4f3e-a5df-bb0688ae15fd","Type":"ContainerStarted","Data":"f2bf0f699b98633d511a2760b163e208920d6d3f785337d26db15c251db5ca33"} Jul 07 15:08:18.214127 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:18.214089 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjt6x" event={"ID":"6e56a8b2-e0b9-4f3e-a5df-bb0688ae15fd","Type":"ContainerStarted","Data":"eef2dd9f90efc81657fb69c9800ef3c5b6a1f7428ffb0ee048fcbcfaa6a70774"} Jul 07 15:08:18.214524 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:18.214172 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjt6x" Jul 07 15:08:29.221370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:29.221342 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjt6x" Jul 07 15:08:29.245165 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:08:29.245113 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cjt6x" podStartSLOduration=13.07892536 podStartE2EDuration="15.245098543s" totalImagesPullingTime="2.166173183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:08:18.236892426 +0000 UTC m=+427.662280815" watchObservedRunningTime="2026-07-07 15:08:29.245098543 +0000 UTC m=+438.670486931" Jul 07 15:10:32.189855 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.189819 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549"] Jul 07 15:10:32.193653 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.193635 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.196650 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.196626 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 15:10:32.198048 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.197995 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tjdd8\"" Jul 07 15:10:32.198180 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.198068 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 15:10:32.198180 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.198003 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 15:10:32.203994 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.203971 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549"] Jul 07 15:10:32.243884 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.243854 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6d462d9-8f38-47cc-bd7e-350fbf623d00-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.244033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.243898 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6d462d9-8f38-47cc-bd7e-350fbf623d00-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.244033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.243918 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6d462d9-8f38-47cc-bd7e-350fbf623d00-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.244033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.243932 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6d462d9-8f38-47cc-bd7e-350fbf623d00-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.244154 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.244039 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srgm2\" (UniqueName: \"kubernetes.io/projected/a6d462d9-8f38-47cc-bd7e-350fbf623d00-kube-api-access-srgm2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.244154 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.244075 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6d462d9-8f38-47cc-bd7e-350fbf623d00-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.345513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.345476 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6d462d9-8f38-47cc-bd7e-350fbf623d00-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.345698 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.345524 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6d462d9-8f38-47cc-bd7e-350fbf623d00-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.345698 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.345543 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6d462d9-8f38-47cc-bd7e-350fbf623d00-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.345698 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.345559 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6d462d9-8f38-47cc-bd7e-350fbf623d00-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.345698 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.345593 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srgm2\" (UniqueName: \"kubernetes.io/projected/a6d462d9-8f38-47cc-bd7e-350fbf623d00-kube-api-access-srgm2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.345698 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.345613 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6d462d9-8f38-47cc-bd7e-350fbf623d00-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.346003 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.345977 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6d462d9-8f38-47cc-bd7e-350fbf623d00-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.346080 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.346012 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6d462d9-8f38-47cc-bd7e-350fbf623d00-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.346080 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.346058 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6d462d9-8f38-47cc-bd7e-350fbf623d00-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.347807 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.347780 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6d462d9-8f38-47cc-bd7e-350fbf623d00-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.348009 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.347991 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6d462d9-8f38-47cc-bd7e-350fbf623d00-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.354878 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.354858 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srgm2\" (UniqueName: \"kubernetes.io/projected/a6d462d9-8f38-47cc-bd7e-350fbf623d00-kube-api-access-srgm2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549\" (UID: \"a6d462d9-8f38-47cc-bd7e-350fbf623d00\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.505731 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.505647 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:32.645909 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.645882 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549"] Jul 07 15:10:32.647178 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:10:32.647149 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d462d9_8f38_47cc_bd7e_350fbf623d00.slice/crio-19b9fb9ed4ee78f1003fde1b7ba3d5ebe95577bfa21b07fe6fbcacc2ef7597a2 WatchSource:0}: Error finding container 19b9fb9ed4ee78f1003fde1b7ba3d5ebe95577bfa21b07fe6fbcacc2ef7597a2: Status 404 returned error can't find the container with id 19b9fb9ed4ee78f1003fde1b7ba3d5ebe95577bfa21b07fe6fbcacc2ef7597a2 Jul 07 15:10:32.889258 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:32.889170 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" event={"ID":"a6d462d9-8f38-47cc-bd7e-350fbf623d00","Type":"ContainerStarted","Data":"19b9fb9ed4ee78f1003fde1b7ba3d5ebe95577bfa21b07fe6fbcacc2ef7597a2"} Jul 07 15:10:33.903432 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:33.903400 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" event={"ID":"a6d462d9-8f38-47cc-bd7e-350fbf623d00","Type":"ContainerStarted","Data":"fce3b0cd7bc9c1b43bd01fc2ba2f9fc7ef10c9d8c806801ecae156c1eac7459c"} Jul 07 15:10:33.922286 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:33.922232 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" podStartSLOduration=0.760807122 podStartE2EDuration="1.922217022s" totalImagesPullingTime="1.1614099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:10:33.921527208 +0000 UTC m=+563.346915597" watchObservedRunningTime="2026-07-07 15:10:33.922217022 +0000 UTC m=+563.347605409" Jul 07 15:10:42.506386 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:42.506352 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:42.506786 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:42.506442 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:42.519446 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:42.519422 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:10:42.958692 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:10:42.958666 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549" Jul 07 15:11:08.385625 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.385592 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd"] Jul 07 15:11:08.388806 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.388787 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.392183 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.392165 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 07 15:11:08.403159 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.403133 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd"] Jul 07 15:11:08.480071 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.480031 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9031ede5-3faa-492e-912b-a2d4b15d9cba-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.480236 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.480089 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9031ede5-3faa-492e-912b-a2d4b15d9cba-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.480236 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.480148 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9031ede5-3faa-492e-912b-a2d4b15d9cba-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.480236 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.480188 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9031ede5-3faa-492e-912b-a2d4b15d9cba-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.480236 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.480222 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j84qm\" (UniqueName: \"kubernetes.io/projected/9031ede5-3faa-492e-912b-a2d4b15d9cba-kube-api-access-j84qm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.480454 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.480243 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9031ede5-3faa-492e-912b-a2d4b15d9cba-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.581546 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.581506 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9031ede5-3faa-492e-912b-a2d4b15d9cba-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.581719 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.581561 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9031ede5-3faa-492e-912b-a2d4b15d9cba-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.581719 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.581600 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9031ede5-3faa-492e-912b-a2d4b15d9cba-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.581719 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.581646 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j84qm\" (UniqueName: \"kubernetes.io/projected/9031ede5-3faa-492e-912b-a2d4b15d9cba-kube-api-access-j84qm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.581719 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.581675 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9031ede5-3faa-492e-912b-a2d4b15d9cba-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.581922 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.581735 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9031ede5-3faa-492e-912b-a2d4b15d9cba-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.582024 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.582002 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9031ede5-3faa-492e-912b-a2d4b15d9cba-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.582109 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.582088 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9031ede5-3faa-492e-912b-a2d4b15d9cba-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.582109 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.582105 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9031ede5-3faa-492e-912b-a2d4b15d9cba-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.584030 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.584007 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9031ede5-3faa-492e-912b-a2d4b15d9cba-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.584251 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.584231 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9031ede5-3faa-492e-912b-a2d4b15d9cba-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.591219 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.591195 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j84qm\" (UniqueName: \"kubernetes.io/projected/9031ede5-3faa-492e-912b-a2d4b15d9cba-kube-api-access-j84qm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9xdd\" (UID: \"9031ede5-3faa-492e-912b-a2d4b15d9cba\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.698917 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.698876 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:08.835510 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:08.835485 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd"] Jul 07 15:11:08.837862 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:11:08.837832 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9031ede5_3faa_492e_912b_a2d4b15d9cba.slice/crio-a4743d7c8f37896170c91bceea1f289cb4ff7cf00f66a2a83f6a545dce94c5b3 WatchSource:0}: Error finding container a4743d7c8f37896170c91bceea1f289cb4ff7cf00f66a2a83f6a545dce94c5b3: Status 404 returned error can't find the container with id a4743d7c8f37896170c91bceea1f289cb4ff7cf00f66a2a83f6a545dce94c5b3 Jul 07 15:11:09.073256 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:09.073230 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" event={"ID":"9031ede5-3faa-492e-912b-a2d4b15d9cba","Type":"ContainerStarted","Data":"a4743d7c8f37896170c91bceea1f289cb4ff7cf00f66a2a83f6a545dce94c5b3"} Jul 07 15:11:10.079747 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:10.079713 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" event={"ID":"9031ede5-3faa-492e-912b-a2d4b15d9cba","Type":"ContainerStarted","Data":"0ffbd0f4b62552bd2aa553c16df95ec27e37f3e75830d60221ad3eb6d6ae08e2"} Jul 07 15:11:10.103424 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:10.103371 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" podStartSLOduration=1.875026665 podStartE2EDuration="2.103354227s" totalImagesPullingTime="228.327562ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:11:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:11:10.101342728 +0000 UTC m=+599.526731116" watchObservedRunningTime="2026-07-07 15:11:10.103354227 +0000 UTC m=+599.528742638" Jul 07 15:11:11.003199 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:11.003170 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:11:11.005290 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:11.005265 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:11:18.699704 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:18.699662 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:18.700184 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:18.699735 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:18.712113 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:18.712085 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:11:19.138864 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:11:19.138779 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9xdd" Jul 07 15:16:11.036057 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:16:11.035981 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:16:11.036543 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:16:11.036268 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:21:11.072222 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:21:11.072185 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:21:11.073445 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:21:11.073424 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:26:11.105587 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:26:11.105554 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:26:11.109139 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:26:11.109122 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:29:43.098341 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.098262 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk"] Jul 07 15:29:43.102210 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.102184 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.105089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.105062 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c55c8ec1\"/\"openshift-service-ca.crt\"" Jul 07 15:29:43.106160 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.106133 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c55c8ec1\"/\"e2e-lifecycle-model-c55c8ec1-kserve-self-signed-certs\"" Jul 07 15:29:43.106291 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.106186 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c55c8ec1\"/\"kube-root-ca.crt\"" Jul 07 15:29:43.106291 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.106210 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c55c8ec1\"/\"default-dockercfg-7w6ms\"" Jul 07 15:29:43.113222 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.113204 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk"] Jul 07 15:29:43.242098 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.242061 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcq2t\" (UniqueName: \"kubernetes.io/projected/b25ce6a2-7635-4170-a60d-5a527e4ce448-kube-api-access-rcq2t\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.242305 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.242108 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-dshm\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.242305 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.242133 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-tmp-dir\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.242305 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.242148 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b25ce6a2-7635-4170-a60d-5a527e4ce448-tls-certs\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.242305 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.242204 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-home\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.242467 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.242301 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-model-cache\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.343087 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.343049 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcq2t\" (UniqueName: \"kubernetes.io/projected/b25ce6a2-7635-4170-a60d-5a527e4ce448-kube-api-access-rcq2t\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.343270 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.343096 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-dshm\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.343270 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.343212 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-tmp-dir\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.343270 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.343255 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b25ce6a2-7635-4170-a60d-5a527e4ce448-tls-certs\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.343438 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.343307 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-home\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.343438 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.343375 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-model-cache\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.343579 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.343561 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-tmp-dir\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.343674 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.343654 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-home\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.343736 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.343710 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-model-cache\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.345203 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.345181 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-dshm\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.345775 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.345756 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b25ce6a2-7635-4170-a60d-5a527e4ce448-tls-certs\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.351996 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.351899 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcq2t\" (UniqueName: \"kubernetes.io/projected/b25ce6a2-7635-4170-a60d-5a527e4ce448-kube-api-access-rcq2t\") pod \"e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.413220 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.413181 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:43.558154 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.558104 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk"] Jul 07 15:29:43.563601 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:29:43.563570 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb25ce6a2_7635_4170_a60d_5a527e4ce448.slice/crio-487798ca0a83bac2385e232b359606f978d567a1bcbe3475e334fb579e9db086 WatchSource:0}: Error finding container 487798ca0a83bac2385e232b359606f978d567a1bcbe3475e334fb579e9db086: Status 404 returned error can't find the container with id 487798ca0a83bac2385e232b359606f978d567a1bcbe3475e334fb579e9db086 Jul 07 15:29:43.678619 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.678586 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" event={"ID":"b25ce6a2-7635-4170-a60d-5a527e4ce448","Type":"ContainerStarted","Data":"42e034f04796815dea4c908a2424bba7ddddb57db22fd2f15809043dccac2bf2"} Jul 07 15:29:43.678619 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.678626 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" event={"ID":"b25ce6a2-7635-4170-a60d-5a527e4ce448","Type":"ContainerStarted","Data":"487798ca0a83bac2385e232b359606f978d567a1bcbe3475e334fb579e9db086"} Jul 07 15:29:43.700221 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:43.700163 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" podStartSLOduration=0.700144513 podStartE2EDuration="700.144513ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:29:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:29:43.698916047 +0000 UTC m=+1713.124304417" watchObservedRunningTime="2026-07-07 15:29:43.700144513 +0000 UTC m=+1713.125532930" Jul 07 15:29:53.413426 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:53.413392 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:53.413816 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:53.413477 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:53.426021 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:53.426000 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:29:53.741079 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:29:53.741053 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:30:00.147600 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:00.147564 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723970-d5bxv"] Jul 07 15:30:00.151447 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:00.151424 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723970-d5bxv" Jul 07 15:30:00.154854 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:00.154833 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-l6lq8\"" Jul 07 15:30:00.164457 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:00.164437 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723970-d5bxv"] Jul 07 15:30:00.202444 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:00.202405 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkfkn\" (UniqueName: \"kubernetes.io/projected/9526c65e-7436-41dc-8a8b-afe7a07b0dad-kube-api-access-bkfkn\") pod \"maas-api-key-cleanup-29723970-d5bxv\" (UID: \"9526c65e-7436-41dc-8a8b-afe7a07b0dad\") " pod="opendatahub/maas-api-key-cleanup-29723970-d5bxv" Jul 07 15:30:00.302910 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:00.302868 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkfkn\" (UniqueName: \"kubernetes.io/projected/9526c65e-7436-41dc-8a8b-afe7a07b0dad-kube-api-access-bkfkn\") pod \"maas-api-key-cleanup-29723970-d5bxv\" (UID: \"9526c65e-7436-41dc-8a8b-afe7a07b0dad\") " pod="opendatahub/maas-api-key-cleanup-29723970-d5bxv" Jul 07 15:30:00.314963 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:00.314902 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkfkn\" (UniqueName: \"kubernetes.io/projected/9526c65e-7436-41dc-8a8b-afe7a07b0dad-kube-api-access-bkfkn\") pod \"maas-api-key-cleanup-29723970-d5bxv\" (UID: \"9526c65e-7436-41dc-8a8b-afe7a07b0dad\") " pod="opendatahub/maas-api-key-cleanup-29723970-d5bxv" Jul 07 15:30:00.462157 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:00.462119 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723970-d5bxv" Jul 07 15:30:00.611012 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:30:00.610976 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9526c65e_7436_41dc_8a8b_afe7a07b0dad.slice/crio-b132752b5f39a18df3ffd0b2430facd11b88fe1f9eab534ef9cb7796e585f221 WatchSource:0}: Error finding container b132752b5f39a18df3ffd0b2430facd11b88fe1f9eab534ef9cb7796e585f221: Status 404 returned error can't find the container with id b132752b5f39a18df3ffd0b2430facd11b88fe1f9eab534ef9cb7796e585f221 Jul 07 15:30:00.611896 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:00.611867 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723970-d5bxv"] Jul 07 15:30:00.613331 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:00.613313 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:30:00.767394 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:00.767313 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723970-d5bxv" event={"ID":"9526c65e-7436-41dc-8a8b-afe7a07b0dad","Type":"ContainerStarted","Data":"b132752b5f39a18df3ffd0b2430facd11b88fe1f9eab534ef9cb7796e585f221"} Jul 07 15:30:03.785006 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:03.784970 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723970-d5bxv" event={"ID":"9526c65e-7436-41dc-8a8b-afe7a07b0dad","Type":"ContainerStarted","Data":"4582542891fa3e538430eed36d4a6fe777334fe151215fe5bc39186f52759963"} Jul 07 15:30:03.802979 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:03.802905 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723970-d5bxv" podStartSLOduration=1.504563184 podStartE2EDuration="3.802887788s" totalImagesPullingTime="2.298324604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:30:03.800613249 +0000 UTC m=+1733.226001638" watchObservedRunningTime="2026-07-07 15:30:03.802887788 +0000 UTC m=+1733.228276394" Jul 07 15:30:04.791566 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:04.791529 2565 generic.go:356] "Generic (PLEG): container finished" podID="9526c65e-7436-41dc-8a8b-afe7a07b0dad" containerID="4582542891fa3e538430eed36d4a6fe777334fe151215fe5bc39186f52759963" exitCode=0 Jul 07 15:30:04.791566 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:04.791568 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723970-d5bxv" event={"ID":"9526c65e-7436-41dc-8a8b-afe7a07b0dad","Type":"ContainerDied","Data":"4582542891fa3e538430eed36d4a6fe777334fe151215fe5bc39186f52759963"} Jul 07 15:30:05.930637 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:05.930612 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723970-d5bxv" Jul 07 15:30:06.056545 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:06.056459 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkfkn\" (UniqueName: \"kubernetes.io/projected/9526c65e-7436-41dc-8a8b-afe7a07b0dad-kube-api-access-bkfkn\") pod \"9526c65e-7436-41dc-8a8b-afe7a07b0dad\" (UID: \"9526c65e-7436-41dc-8a8b-afe7a07b0dad\") " Jul 07 15:30:06.058702 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:06.058672 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9526c65e-7436-41dc-8a8b-afe7a07b0dad-kube-api-access-bkfkn" (OuterVolumeSpecName: "kube-api-access-bkfkn") pod "9526c65e-7436-41dc-8a8b-afe7a07b0dad" (UID: "9526c65e-7436-41dc-8a8b-afe7a07b0dad"). InnerVolumeSpecName "kube-api-access-bkfkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:30:06.157714 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:06.157678 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkfkn\" (UniqueName: \"kubernetes.io/projected/9526c65e-7436-41dc-8a8b-afe7a07b0dad-kube-api-access-bkfkn\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:30:06.803192 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:06.803158 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723970-d5bxv" event={"ID":"9526c65e-7436-41dc-8a8b-afe7a07b0dad","Type":"ContainerDied","Data":"b132752b5f39a18df3ffd0b2430facd11b88fe1f9eab534ef9cb7796e585f221"} Jul 07 15:30:06.803354 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:06.803211 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b132752b5f39a18df3ffd0b2430facd11b88fe1f9eab534ef9cb7796e585f221" Jul 07 15:30:06.803354 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:06.803171 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723970-d5bxv" Jul 07 15:30:20.001821 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:20.001788 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk"] Jul 07 15:30:35.022121 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.022079 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" podUID="b25ce6a2-7635-4170-a60d-5a527e4ce448" containerName="main" containerID="cri-o://42e034f04796815dea4c908a2424bba7ddddb57db22fd2f15809043dccac2bf2" gracePeriod=45 Jul 07 15:30:35.279424 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.279368 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:30:35.312754 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.312724 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-model-cache\") pod \"b25ce6a2-7635-4170-a60d-5a527e4ce448\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " Jul 07 15:30:35.312959 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.312767 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-dshm\") pod \"b25ce6a2-7635-4170-a60d-5a527e4ce448\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " Jul 07 15:30:35.312959 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.312828 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcq2t\" (UniqueName: \"kubernetes.io/projected/b25ce6a2-7635-4170-a60d-5a527e4ce448-kube-api-access-rcq2t\") pod \"b25ce6a2-7635-4170-a60d-5a527e4ce448\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " Jul 07 15:30:35.312959 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.312850 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-tmp-dir\") pod \"b25ce6a2-7635-4170-a60d-5a527e4ce448\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " Jul 07 15:30:35.312959 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.312890 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b25ce6a2-7635-4170-a60d-5a527e4ce448-tls-certs\") pod \"b25ce6a2-7635-4170-a60d-5a527e4ce448\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " Jul 07 15:30:35.312959 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.312922 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-home\") pod \"b25ce6a2-7635-4170-a60d-5a527e4ce448\" (UID: \"b25ce6a2-7635-4170-a60d-5a527e4ce448\") " Jul 07 15:30:35.313235 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.313053 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-model-cache" (OuterVolumeSpecName: "model-cache") pod "b25ce6a2-7635-4170-a60d-5a527e4ce448" (UID: "b25ce6a2-7635-4170-a60d-5a527e4ce448"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:35.313396 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.313365 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-home" (OuterVolumeSpecName: "home") pod "b25ce6a2-7635-4170-a60d-5a527e4ce448" (UID: "b25ce6a2-7635-4170-a60d-5a527e4ce448"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:35.313620 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.313601 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-home\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:30:35.313679 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.313626 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-model-cache\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:30:35.313715 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.313695 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b25ce6a2-7635-4170-a60d-5a527e4ce448" (UID: "b25ce6a2-7635-4170-a60d-5a527e4ce448"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:35.315220 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.315196 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-dshm" (OuterVolumeSpecName: "dshm") pod "b25ce6a2-7635-4170-a60d-5a527e4ce448" (UID: "b25ce6a2-7635-4170-a60d-5a527e4ce448"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:35.315427 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.315400 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25ce6a2-7635-4170-a60d-5a527e4ce448-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b25ce6a2-7635-4170-a60d-5a527e4ce448" (UID: "b25ce6a2-7635-4170-a60d-5a527e4ce448"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:30:35.315427 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.315401 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25ce6a2-7635-4170-a60d-5a527e4ce448-kube-api-access-rcq2t" (OuterVolumeSpecName: "kube-api-access-rcq2t") pod "b25ce6a2-7635-4170-a60d-5a527e4ce448" (UID: "b25ce6a2-7635-4170-a60d-5a527e4ce448"). InnerVolumeSpecName "kube-api-access-rcq2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:30:35.415030 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.415000 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcq2t\" (UniqueName: \"kubernetes.io/projected/b25ce6a2-7635-4170-a60d-5a527e4ce448-kube-api-access-rcq2t\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:30:35.415030 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.415027 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-tmp-dir\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:30:35.415030 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.415040 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b25ce6a2-7635-4170-a60d-5a527e4ce448-tls-certs\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:30:35.415256 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.415049 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b25ce6a2-7635-4170-a60d-5a527e4ce448-dshm\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:30:35.952763 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.952727 2565 generic.go:356] "Generic (PLEG): container finished" podID="b25ce6a2-7635-4170-a60d-5a527e4ce448" containerID="42e034f04796815dea4c908a2424bba7ddddb57db22fd2f15809043dccac2bf2" exitCode=0 Jul 07 15:30:35.952959 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.952773 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" Jul 07 15:30:35.952959 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.952787 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" event={"ID":"b25ce6a2-7635-4170-a60d-5a527e4ce448","Type":"ContainerDied","Data":"42e034f04796815dea4c908a2424bba7ddddb57db22fd2f15809043dccac2bf2"} Jul 07 15:30:35.952959 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.952825 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk" event={"ID":"b25ce6a2-7635-4170-a60d-5a527e4ce448","Type":"ContainerDied","Data":"487798ca0a83bac2385e232b359606f978d567a1bcbe3475e334fb579e9db086"} Jul 07 15:30:35.952959 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.952848 2565 scope.go:117] "RemoveContainer" containerID="42e034f04796815dea4c908a2424bba7ddddb57db22fd2f15809043dccac2bf2" Jul 07 15:30:35.962467 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.962450 2565 scope.go:117] "RemoveContainer" containerID="42e034f04796815dea4c908a2424bba7ddddb57db22fd2f15809043dccac2bf2" Jul 07 15:30:35.962706 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:30:35.962689 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42e034f04796815dea4c908a2424bba7ddddb57db22fd2f15809043dccac2bf2\": container with ID starting with 42e034f04796815dea4c908a2424bba7ddddb57db22fd2f15809043dccac2bf2 not found: ID does not exist" containerID="42e034f04796815dea4c908a2424bba7ddddb57db22fd2f15809043dccac2bf2" Jul 07 15:30:35.962756 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.962714 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42e034f04796815dea4c908a2424bba7ddddb57db22fd2f15809043dccac2bf2"} err="failed to get container status \"42e034f04796815dea4c908a2424bba7ddddb57db22fd2f15809043dccac2bf2\": rpc error: code = NotFound desc = could not find container \"42e034f04796815dea4c908a2424bba7ddddb57db22fd2f15809043dccac2bf2\": container with ID starting with 42e034f04796815dea4c908a2424bba7ddddb57db22fd2f15809043dccac2bf2 not found: ID does not exist" Jul 07 15:30:35.974595 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.974567 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk"] Jul 07 15:30:35.978746 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:35.978727 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-c55c8ec1/e2e-lifecycle-model-c55c8ec1-kserve-b5dcb559f-9blqk"] Jul 07 15:30:37.104915 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:30:37.104882 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b25ce6a2-7635-4170-a60d-5a527e4ce448" path="/var/lib/kubelet/pods/b25ce6a2-7635-4170-a60d-5a527e4ce448/volumes" Jul 07 15:31:11.138687 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:31:11.138656 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:31:11.148112 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:31:11.148083 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:32:09.940189 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:09.940149 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84"] Jul 07 15:32:09.940691 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:09.940671 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9526c65e-7436-41dc-8a8b-afe7a07b0dad" containerName="cleanup" Jul 07 15:32:09.940743 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:09.940695 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9526c65e-7436-41dc-8a8b-afe7a07b0dad" containerName="cleanup" Jul 07 15:32:09.940743 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:09.940708 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b25ce6a2-7635-4170-a60d-5a527e4ce448" containerName="main" Jul 07 15:32:09.940743 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:09.940716 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b25ce6a2-7635-4170-a60d-5a527e4ce448" containerName="main" Jul 07 15:32:09.940846 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:09.940796 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b25ce6a2-7635-4170-a60d-5a527e4ce448" containerName="main" Jul 07 15:32:09.940846 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:09.940806 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9526c65e-7436-41dc-8a8b-afe7a07b0dad" containerName="cleanup" Jul 07 15:32:09.943875 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:09.943856 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:09.948200 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:09.947217 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-44430f-openshift-default-dockercfg-7dw48\"" Jul 07 15:32:09.960524 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:09.960497 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84"] Jul 07 15:32:10.048737 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.048703 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-workload-socket\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.048965 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.048752 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-data\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.048965 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.048791 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-credential-socket\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.048965 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.048809 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-workload-certs\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.048965 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.048838 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-podinfo\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.048965 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.048853 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-envoy\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.048965 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.048910 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flk5c\" (UniqueName: \"kubernetes.io/projected/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-kube-api-access-flk5c\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.049383 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.049023 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-token\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.049383 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.049053 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istiod-ca-cert\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.150212 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.150174 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-token\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.150212 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.150218 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istiod-ca-cert\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.150475 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.150241 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-workload-socket\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.150475 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.150266 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-data\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.150475 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.150297 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-credential-socket\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.150475 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.150312 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-workload-certs\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.150475 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.150346 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-podinfo\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.150475 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.150369 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-envoy\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.150475 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.150401 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flk5c\" (UniqueName: \"kubernetes.io/projected/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-kube-api-access-flk5c\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.150817 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.150728 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-workload-certs\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.150817 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.150727 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-workload-socket\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.150817 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.150783 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-credential-socket\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.151014 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.150836 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-data\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.151065 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.151035 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istiod-ca-cert\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.153207 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.153168 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-envoy\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.153484 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.153462 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-podinfo\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.160300 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.160270 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-token\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.160410 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.160341 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flk5c\" (UniqueName: \"kubernetes.io/projected/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-kube-api-access-flk5c\") pod \"e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.263796 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.263709 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:10.405299 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.405275 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84"] Jul 07 15:32:10.407325 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:32:10.407300 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae7c380_ff58_40fd_86b8_335ef4bc37ed.slice/crio-1be58766d1bd580663b826bbbe6a71e93ecca0c0042484128e11cf548d3a1964 WatchSource:0}: Error finding container 1be58766d1bd580663b826bbbe6a71e93ecca0c0042484128e11cf548d3a1964: Status 404 returned error can't find the container with id 1be58766d1bd580663b826bbbe6a71e93ecca0c0042484128e11cf548d3a1964 Jul 07 15:32:10.409317 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.409282 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:32:10.409384 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.409363 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:32:10.409425 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.409393 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:32:10.429669 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:10.429636 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" event={"ID":"0ae7c380-ff58-40fd-86b8-335ef4bc37ed","Type":"ContainerStarted","Data":"1be58766d1bd580663b826bbbe6a71e93ecca0c0042484128e11cf548d3a1964"} Jul 07 15:32:11.436882 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:11.436846 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" event={"ID":"0ae7c380-ff58-40fd-86b8-335ef4bc37ed","Type":"ContainerStarted","Data":"7c157cbda537ab1dfa03cd9803e714175d2aecabcbed3918a44518086046a514"} Jul 07 15:32:11.465229 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:11.465178 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" podStartSLOduration=2.465162409 podStartE2EDuration="2.465162409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:32:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:32:11.46243624 +0000 UTC m=+1860.887824634" watchObservedRunningTime="2026-07-07 15:32:11.465162409 +0000 UTC m=+1860.890551097" Jul 07 15:32:12.264589 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:12.264550 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:12.269808 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:12.269781 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:12.442852 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:12.442821 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:12.443834 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:12.443810 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:27.657761 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:27.657707 2565 patch_prober.go:28] interesting pod/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:59058->10.134.0.40:15021: read: connection reset by peer" start-of-body= Jul 07 15:32:27.658183 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:27.657773 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" podUID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:59058->10.134.0.40:15021: read: connection reset by peer" Jul 07 15:32:28.536153 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:28.536119 2565 generic.go:356] "Generic (PLEG): container finished" podID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerID="7c157cbda537ab1dfa03cd9803e714175d2aecabcbed3918a44518086046a514" exitCode=0 Jul 07 15:32:28.536336 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:28.536192 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" event={"ID":"0ae7c380-ff58-40fd-86b8-335ef4bc37ed","Type":"ContainerDied","Data":"7c157cbda537ab1dfa03cd9803e714175d2aecabcbed3918a44518086046a514"} Jul 07 15:32:28.536652 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:28.536635 2565 scope.go:117] "RemoveContainer" containerID="7c157cbda537ab1dfa03cd9803e714175d2aecabcbed3918a44518086046a514" Jul 07 15:32:28.538186 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:28.537884 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:32:28.538186 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:28.537993 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:32:28.538186 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:28.538077 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:32:29.542400 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:29.542363 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" event={"ID":"0ae7c380-ff58-40fd-86b8-335ef4bc37ed","Type":"ContainerStarted","Data":"3dd0b48ec777b704beffd0b66f84927fe9a7518b71d574461fbd7e44027af271"} Jul 07 15:32:30.264669 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:30.264632 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:30.265073 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:30.265046 2565 patch_prober.go:28] interesting pod/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 15:32:30.265253 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:30.265226 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" podUID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 15:32:31.264373 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:31.264324 2565 patch_prober.go:28] interesting pod/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 15:32:31.264876 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:31.264406 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" podUID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 15:32:32.265663 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:32.265145 2565 patch_prober.go:28] interesting pod/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 15:32:32.265663 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:32.265212 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" podUID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 15:32:33.265001 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:33.264957 2565 patch_prober.go:28] interesting pod/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 15:32:33.265212 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:33.265026 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" podUID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 15:32:34.268333 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:34.268298 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:34.268700 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:34.268584 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:32:34.269391 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:32:34.269371 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:34:09.687896 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:09.687802 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84"] Jul 07 15:34:09.688799 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:09.688022 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" podUID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerName="istio-proxy" containerID="cri-o://3dd0b48ec777b704beffd0b66f84927fe9a7518b71d574461fbd7e44027af271" gracePeriod=30 Jul 07 15:34:14.931511 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:14.931485 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:34:15.074417 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.074322 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-credential-socket\") pod \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " Jul 07 15:34:15.074417 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.074375 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istiod-ca-cert\") pod \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " Jul 07 15:34:15.074417 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.074396 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-envoy\") pod \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " Jul 07 15:34:15.074696 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.074441 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-workload-socket\") pod \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " Jul 07 15:34:15.074696 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.074558 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-data\") pod \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " Jul 07 15:34:15.074696 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.074585 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-workload-certs\") pod \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " Jul 07 15:34:15.074696 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.074612 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-podinfo\") pod \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " Jul 07 15:34:15.074696 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.074674 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-token\") pod \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " Jul 07 15:34:15.074696 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.074685 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0ae7c380-ff58-40fd-86b8-335ef4bc37ed" (UID: "0ae7c380-ff58-40fd-86b8-335ef4bc37ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:15.075048 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.074704 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flk5c\" (UniqueName: \"kubernetes.io/projected/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-kube-api-access-flk5c\") pod \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\" (UID: \"0ae7c380-ff58-40fd-86b8-335ef4bc37ed\") " Jul 07 15:34:15.075048 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.074851 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0ae7c380-ff58-40fd-86b8-335ef4bc37ed" (UID: "0ae7c380-ff58-40fd-86b8-335ef4bc37ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:15.075048 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.074811 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0ae7c380-ff58-40fd-86b8-335ef4bc37ed" (UID: "0ae7c380-ff58-40fd-86b8-335ef4bc37ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:34:15.075236 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.075142 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istiod-ca-cert\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:15.075236 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.075161 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-workload-certs\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:15.075236 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.075173 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-credential-socket\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:15.075389 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.075290 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0ae7c380-ff58-40fd-86b8-335ef4bc37ed" (UID: "0ae7c380-ff58-40fd-86b8-335ef4bc37ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:15.075558 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.075536 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-data" (OuterVolumeSpecName: "istio-data") pod "0ae7c380-ff58-40fd-86b8-335ef4bc37ed" (UID: "0ae7c380-ff58-40fd-86b8-335ef4bc37ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:15.077056 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.077020 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-kube-api-access-flk5c" (OuterVolumeSpecName: "kube-api-access-flk5c") pod "0ae7c380-ff58-40fd-86b8-335ef4bc37ed" (UID: "0ae7c380-ff58-40fd-86b8-335ef4bc37ed"). InnerVolumeSpecName "kube-api-access-flk5c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:34:15.077056 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.077019 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0ae7c380-ff58-40fd-86b8-335ef4bc37ed" (UID: "0ae7c380-ff58-40fd-86b8-335ef4bc37ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:34:15.077188 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.077091 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-token" (OuterVolumeSpecName: "istio-token") pod "0ae7c380-ff58-40fd-86b8-335ef4bc37ed" (UID: "0ae7c380-ff58-40fd-86b8-335ef4bc37ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:34:15.077188 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.077093 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0ae7c380-ff58-40fd-86b8-335ef4bc37ed" (UID: "0ae7c380-ff58-40fd-86b8-335ef4bc37ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:15.092987 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.092959 2565 generic.go:356] "Generic (PLEG): container finished" podID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerID="3dd0b48ec777b704beffd0b66f84927fe9a7518b71d574461fbd7e44027af271" exitCode=0 Jul 07 15:34:15.092987 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.092969 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" event={"ID":"0ae7c380-ff58-40fd-86b8-335ef4bc37ed","Type":"ContainerDied","Data":"3dd0b48ec777b704beffd0b66f84927fe9a7518b71d574461fbd7e44027af271"} Jul 07 15:34:15.093142 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.093001 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" Jul 07 15:34:15.093142 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.093026 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84" event={"ID":"0ae7c380-ff58-40fd-86b8-335ef4bc37ed","Type":"ContainerDied","Data":"1be58766d1bd580663b826bbbe6a71e93ecca0c0042484128e11cf548d3a1964"} Jul 07 15:34:15.093142 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.093049 2565 scope.go:117] "RemoveContainer" containerID="3dd0b48ec777b704beffd0b66f84927fe9a7518b71d574461fbd7e44027af271" Jul 07 15:34:15.103619 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.102728 2565 scope.go:117] "RemoveContainer" containerID="7c157cbda537ab1dfa03cd9803e714175d2aecabcbed3918a44518086046a514" Jul 07 15:34:15.117512 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.117495 2565 scope.go:117] "RemoveContainer" containerID="3dd0b48ec777b704beffd0b66f84927fe9a7518b71d574461fbd7e44027af271" Jul 07 15:34:15.117768 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:34:15.117748 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dd0b48ec777b704beffd0b66f84927fe9a7518b71d574461fbd7e44027af271\": container with ID starting with 3dd0b48ec777b704beffd0b66f84927fe9a7518b71d574461fbd7e44027af271 not found: ID does not exist" containerID="3dd0b48ec777b704beffd0b66f84927fe9a7518b71d574461fbd7e44027af271" Jul 07 15:34:15.117818 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.117777 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dd0b48ec777b704beffd0b66f84927fe9a7518b71d574461fbd7e44027af271"} err="failed to get container status \"3dd0b48ec777b704beffd0b66f84927fe9a7518b71d574461fbd7e44027af271\": rpc error: code = NotFound desc = could not find container \"3dd0b48ec777b704beffd0b66f84927fe9a7518b71d574461fbd7e44027af271\": container with ID starting with 3dd0b48ec777b704beffd0b66f84927fe9a7518b71d574461fbd7e44027af271 not found: ID does not exist" Jul 07 15:34:15.117818 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.117793 2565 scope.go:117] "RemoveContainer" containerID="7c157cbda537ab1dfa03cd9803e714175d2aecabcbed3918a44518086046a514" Jul 07 15:34:15.118079 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:34:15.118059 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c157cbda537ab1dfa03cd9803e714175d2aecabcbed3918a44518086046a514\": container with ID starting with 7c157cbda537ab1dfa03cd9803e714175d2aecabcbed3918a44518086046a514 not found: ID does not exist" containerID="7c157cbda537ab1dfa03cd9803e714175d2aecabcbed3918a44518086046a514" Jul 07 15:34:15.118134 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.118084 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c157cbda537ab1dfa03cd9803e714175d2aecabcbed3918a44518086046a514"} err="failed to get container status \"7c157cbda537ab1dfa03cd9803e714175d2aecabcbed3918a44518086046a514\": rpc error: code = NotFound desc = could not find container \"7c157cbda537ab1dfa03cd9803e714175d2aecabcbed3918a44518086046a514\": container with ID starting with 7c157cbda537ab1dfa03cd9803e714175d2aecabcbed3918a44518086046a514 not found: ID does not exist" Jul 07 15:34:15.124615 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.124592 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84"] Jul 07 15:34:15.131976 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.131954 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-44430f-openshift-default-6cc87b595c-vkr84"] Jul 07 15:34:15.176022 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.175995 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-token\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:15.176022 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.176019 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flk5c\" (UniqueName: \"kubernetes.io/projected/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-kube-api-access-flk5c\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:15.176174 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.176029 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-envoy\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:15.176174 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.176039 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-workload-socket\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:15.176174 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.176047 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-data\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:15.176174 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:15.176057 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ae7c380-ff58-40fd-86b8-335ef4bc37ed-istio-podinfo\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:17.105594 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:17.105555 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" path="/var/lib/kubelet/pods/0ae7c380-ff58-40fd-86b8-335ef4bc37ed/volumes" Jul 07 15:34:18.876341 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:18.876311 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx"] Jul 07 15:34:18.876824 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:18.876805 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerName="istio-proxy" Jul 07 15:34:18.876902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:18.876827 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerName="istio-proxy" Jul 07 15:34:18.876902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:18.876854 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerName="istio-proxy" Jul 07 15:34:18.876902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:18.876864 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerName="istio-proxy" Jul 07 15:34:18.877087 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:18.876975 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerName="istio-proxy" Jul 07 15:34:18.877087 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:18.876988 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ae7c380-ff58-40fd-86b8-335ef4bc37ed" containerName="istio-proxy" Jul 07 15:34:18.880082 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:18.880060 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:18.883094 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:18.883075 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-8d3dc6-openshift-default-dockercfg-kwjk6\"" Jul 07 15:34:18.892467 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:18.892446 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx"] Jul 07 15:34:19.014711 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.014669 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb251480-7c9e-4e8e-be76-29653cd875ed-istiod-ca-cert\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.014875 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.014743 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-token\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.014875 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.014765 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-workload-certs\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.014875 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.014800 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-workload-socket\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.014875 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.014823 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-data\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.014875 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.014847 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sskxc\" (UniqueName: \"kubernetes.io/projected/fb251480-7c9e-4e8e-be76-29653cd875ed-kube-api-access-sskxc\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.014875 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.014871 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-credential-socket\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.015101 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.014896 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-podinfo\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.015101 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.014931 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-envoy\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.115602 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.115568 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb251480-7c9e-4e8e-be76-29653cd875ed-istiod-ca-cert\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.115783 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.115642 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-token\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.115783 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.115666 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-workload-certs\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.115783 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.115690 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-workload-socket\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.115783 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.115714 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-data\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.115783 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.115748 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sskxc\" (UniqueName: \"kubernetes.io/projected/fb251480-7c9e-4e8e-be76-29653cd875ed-kube-api-access-sskxc\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.115783 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.115774 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-credential-socket\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.116122 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.115798 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-podinfo\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.116122 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.115833 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-envoy\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.116122 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.116082 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-workload-certs\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.116248 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.116158 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-data\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.116430 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.116375 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb251480-7c9e-4e8e-be76-29653cd875ed-istiod-ca-cert\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.116430 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.116381 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-credential-socket\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.116617 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.116589 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-workload-socket\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.118282 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.118263 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-envoy\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.118416 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.118399 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-podinfo\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.128586 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.128529 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-token\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.129284 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.129264 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sskxc\" (UniqueName: \"kubernetes.io/projected/fb251480-7c9e-4e8e-be76-29653cd875ed-kube-api-access-sskxc\") pod \"e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.190056 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.190029 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:19.334845 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.334808 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx"] Jul 07 15:34:19.337994 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:34:19.337969 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb251480_7c9e_4e8e_be76_29653cd875ed.slice/crio-b2be22f21ff4a5af4c14f6de909de4164b24f9dd6fc8b72fb086de83bba50ec0 WatchSource:0}: Error finding container b2be22f21ff4a5af4c14f6de909de4164b24f9dd6fc8b72fb086de83bba50ec0: Status 404 returned error can't find the container with id b2be22f21ff4a5af4c14f6de909de4164b24f9dd6fc8b72fb086de83bba50ec0 Jul 07 15:34:19.339995 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.339966 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:34:19.340073 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.340024 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:34:19.340073 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:19.340052 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:34:20.123981 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:20.123950 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" event={"ID":"fb251480-7c9e-4e8e-be76-29653cd875ed","Type":"ContainerStarted","Data":"95f3267d03659c374c0f740f168cbf350247ea442174a35757cc20a8e268a543"} Jul 07 15:34:20.124356 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:20.124008 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" event={"ID":"fb251480-7c9e-4e8e-be76-29653cd875ed","Type":"ContainerStarted","Data":"b2be22f21ff4a5af4c14f6de909de4164b24f9dd6fc8b72fb086de83bba50ec0"} Jul 07 15:34:20.150604 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:20.150555 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" podStartSLOduration=2.150538943 podStartE2EDuration="2.150538943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:34:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:34:20.148258237 +0000 UTC m=+1989.573646646" watchObservedRunningTime="2026-07-07 15:34:20.150538943 +0000 UTC m=+1989.575927330" Jul 07 15:34:20.190574 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:20.190534 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:20.195395 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:20.195367 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:21.128558 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:21.128526 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:21.129540 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:21.129520 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:22.302513 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:22.302477 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj"] Jul 07 15:34:22.304913 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:22.304890 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:22.307838 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:22.307803 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-8d3dc6-serving-cert\"" Jul 07 15:34:22.307983 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:22.307904 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-l6lq8\"" Jul 07 15:34:22.319532 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:22.319509 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj"] Jul 07 15:34:22.448910 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:22.448873 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/661f6abc-0189-462a-bb21-54dbc46bbf97-openshift-service-ca\") pod \"maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj\" (UID: \"661f6abc-0189-462a-bb21-54dbc46bbf97\") " pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:22.449076 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:22.448921 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j4cs\" (UniqueName: \"kubernetes.io/projected/661f6abc-0189-462a-bb21-54dbc46bbf97-kube-api-access-7j4cs\") pod \"maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj\" (UID: \"661f6abc-0189-462a-bb21-54dbc46bbf97\") " pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:22.449076 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:22.449021 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/661f6abc-0189-462a-bb21-54dbc46bbf97-maas-api-tls\") pod \"maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj\" (UID: \"661f6abc-0189-462a-bb21-54dbc46bbf97\") " pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:22.549997 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:22.549962 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/661f6abc-0189-462a-bb21-54dbc46bbf97-maas-api-tls\") pod \"maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj\" (UID: \"661f6abc-0189-462a-bb21-54dbc46bbf97\") " pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:22.550187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:22.550051 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/661f6abc-0189-462a-bb21-54dbc46bbf97-openshift-service-ca\") pod \"maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj\" (UID: \"661f6abc-0189-462a-bb21-54dbc46bbf97\") " pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:22.550187 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:22.550089 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j4cs\" (UniqueName: \"kubernetes.io/projected/661f6abc-0189-462a-bb21-54dbc46bbf97-kube-api-access-7j4cs\") pod \"maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj\" (UID: \"661f6abc-0189-462a-bb21-54dbc46bbf97\") " pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:22.550187 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:34:22.550112 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-8d3dc6-serving-cert: secret "maas-api-e2e-api-b-8d3dc6-serving-cert" not found Jul 07 15:34:22.550187 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:34:22.550176 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/661f6abc-0189-462a-bb21-54dbc46bbf97-maas-api-tls podName:661f6abc-0189-462a-bb21-54dbc46bbf97 nodeName:}" failed. No retries permitted until 2026-07-07 15:34:23.05015905 +0000 UTC m=+1992.475547416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/661f6abc-0189-462a-bb21-54dbc46bbf97-maas-api-tls") pod "maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" (UID: "661f6abc-0189-462a-bb21-54dbc46bbf97") : secret "maas-api-e2e-api-b-8d3dc6-serving-cert" not found Jul 07 15:34:22.550734 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:22.550715 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/661f6abc-0189-462a-bb21-54dbc46bbf97-openshift-service-ca\") pod \"maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj\" (UID: \"661f6abc-0189-462a-bb21-54dbc46bbf97\") " pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:22.564543 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:22.564473 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j4cs\" (UniqueName: \"kubernetes.io/projected/661f6abc-0189-462a-bb21-54dbc46bbf97-kube-api-access-7j4cs\") pod \"maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj\" (UID: \"661f6abc-0189-462a-bb21-54dbc46bbf97\") " pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:23.055311 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:23.055272 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/661f6abc-0189-462a-bb21-54dbc46bbf97-maas-api-tls\") pod \"maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj\" (UID: \"661f6abc-0189-462a-bb21-54dbc46bbf97\") " pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:23.057677 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:23.057649 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/661f6abc-0189-462a-bb21-54dbc46bbf97-maas-api-tls\") pod \"maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj\" (UID: \"661f6abc-0189-462a-bb21-54dbc46bbf97\") " pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:23.215993 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:23.215961 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:23.348854 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:23.348826 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj"] Jul 07 15:34:23.350672 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:34:23.350645 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod661f6abc_0189_462a_bb21_54dbc46bbf97.slice/crio-1c0ad995a2da2ed46107c6ae27f67400c0835442cbacaac9ed4c80e1b7c70602 WatchSource:0}: Error finding container 1c0ad995a2da2ed46107c6ae27f67400c0835442cbacaac9ed4c80e1b7c70602: Status 404 returned error can't find the container with id 1c0ad995a2da2ed46107c6ae27f67400c0835442cbacaac9ed4c80e1b7c70602 Jul 07 15:34:24.145375 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:24.145338 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" event={"ID":"661f6abc-0189-462a-bb21-54dbc46bbf97","Type":"ContainerStarted","Data":"1c0ad995a2da2ed46107c6ae27f67400c0835442cbacaac9ed4c80e1b7c70602"} Jul 07 15:34:26.161888 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:26.161844 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" event={"ID":"661f6abc-0189-462a-bb21-54dbc46bbf97","Type":"ContainerStarted","Data":"6e4f727e21a2f0542e8c3773af1c8892dfe933af92fa2d4c0cbd5f7e89d595fb"} Jul 07 15:34:26.162383 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:26.162000 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:26.183456 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:26.183409 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" podStartSLOduration=2.20928818 podStartE2EDuration="4.183394727s" totalImagesPullingTime="1.974106547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:34:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:34:26.181641912 +0000 UTC m=+1995.607030292" watchObservedRunningTime="2026-07-07 15:34:26.183394727 +0000 UTC m=+1995.608783114" Jul 07 15:34:32.178436 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:32.178404 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:36.130577 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:36.130479 2565 patch_prober.go:28] interesting pod/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 07 15:34:36.130577 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:36.130538 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" podUID="fb251480-7c9e-4e8e-be76-29653cd875ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 07 15:34:36.225606 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:36.225566 2565 generic.go:356] "Generic (PLEG): container finished" podID="fb251480-7c9e-4e8e-be76-29653cd875ed" containerID="95f3267d03659c374c0f740f168cbf350247ea442174a35757cc20a8e268a543" exitCode=0 Jul 07 15:34:36.225606 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:36.225608 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" event={"ID":"fb251480-7c9e-4e8e-be76-29653cd875ed","Type":"ContainerDied","Data":"95f3267d03659c374c0f740f168cbf350247ea442174a35757cc20a8e268a543"} Jul 07 15:34:36.226116 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:36.226098 2565 scope.go:117] "RemoveContainer" containerID="95f3267d03659c374c0f740f168cbf350247ea442174a35757cc20a8e268a543" Jul 07 15:34:36.227182 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:36.227147 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:34:36.227252 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:36.227228 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:34:36.227295 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:36.227264 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 15:34:37.232545 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:37.232511 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" event={"ID":"fb251480-7c9e-4e8e-be76-29653cd875ed","Type":"ContainerStarted","Data":"5bbc69014d4f723e44288b7cef4da7b9894e31becb7b4f5c27b9a9341c18d185"} Jul 07 15:34:38.191119 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:38.191081 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:38.191296 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:38.191266 2565 patch_prober.go:28] interesting pod/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 07 15:34:38.191343 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:38.191312 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" podUID="fb251480-7c9e-4e8e-be76-29653cd875ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 07 15:34:39.190840 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:39.190801 2565 patch_prober.go:28] interesting pod/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 07 15:34:39.191270 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:39.190856 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" podUID="fb251480-7c9e-4e8e-be76-29653cd875ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 07 15:34:40.191263 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:40.191228 2565 patch_prober.go:28] interesting pod/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 07 15:34:40.191653 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:40.191287 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" podUID="fb251480-7c9e-4e8e-be76-29653cd875ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 07 15:34:41.194118 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:41.194084 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:41.194477 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:41.194445 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:41.195113 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:41.195098 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:43.656764 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:43.656731 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj"] Jul 07 15:34:43.657365 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:43.656988 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" podUID="661f6abc-0189-462a-bb21-54dbc46bbf97" containerName="maas-api" containerID="cri-o://6e4f727e21a2f0542e8c3773af1c8892dfe933af92fa2d4c0cbd5f7e89d595fb" gracePeriod=30 Jul 07 15:34:43.908400 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:43.908333 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:43.951792 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:43.951764 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/661f6abc-0189-462a-bb21-54dbc46bbf97-openshift-service-ca\") pod \"661f6abc-0189-462a-bb21-54dbc46bbf97\" (UID: \"661f6abc-0189-462a-bb21-54dbc46bbf97\") " Jul 07 15:34:43.951975 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:43.951806 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j4cs\" (UniqueName: \"kubernetes.io/projected/661f6abc-0189-462a-bb21-54dbc46bbf97-kube-api-access-7j4cs\") pod \"661f6abc-0189-462a-bb21-54dbc46bbf97\" (UID: \"661f6abc-0189-462a-bb21-54dbc46bbf97\") " Jul 07 15:34:43.951975 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:43.951867 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/661f6abc-0189-462a-bb21-54dbc46bbf97-maas-api-tls\") pod \"661f6abc-0189-462a-bb21-54dbc46bbf97\" (UID: \"661f6abc-0189-462a-bb21-54dbc46bbf97\") " Jul 07 15:34:43.952212 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:43.952193 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/661f6abc-0189-462a-bb21-54dbc46bbf97-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "661f6abc-0189-462a-bb21-54dbc46bbf97" (UID: "661f6abc-0189-462a-bb21-54dbc46bbf97"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:34:43.954066 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:43.954037 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/661f6abc-0189-462a-bb21-54dbc46bbf97-kube-api-access-7j4cs" (OuterVolumeSpecName: "kube-api-access-7j4cs") pod "661f6abc-0189-462a-bb21-54dbc46bbf97" (UID: "661f6abc-0189-462a-bb21-54dbc46bbf97"). InnerVolumeSpecName "kube-api-access-7j4cs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:34:43.954188 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:43.954153 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/661f6abc-0189-462a-bb21-54dbc46bbf97-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "661f6abc-0189-462a-bb21-54dbc46bbf97" (UID: "661f6abc-0189-462a-bb21-54dbc46bbf97"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:34:44.052822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:44.052786 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/661f6abc-0189-462a-bb21-54dbc46bbf97-maas-api-tls\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:44.052822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:44.052817 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/661f6abc-0189-462a-bb21-54dbc46bbf97-openshift-service-ca\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:44.052822 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:44.052826 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j4cs\" (UniqueName: \"kubernetes.io/projected/661f6abc-0189-462a-bb21-54dbc46bbf97-kube-api-access-7j4cs\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:44.274062 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:44.274022 2565 generic.go:356] "Generic (PLEG): container finished" podID="661f6abc-0189-462a-bb21-54dbc46bbf97" containerID="6e4f727e21a2f0542e8c3773af1c8892dfe933af92fa2d4c0cbd5f7e89d595fb" exitCode=0 Jul 07 15:34:44.274233 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:44.274076 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" Jul 07 15:34:44.274233 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:44.274107 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" event={"ID":"661f6abc-0189-462a-bb21-54dbc46bbf97","Type":"ContainerDied","Data":"6e4f727e21a2f0542e8c3773af1c8892dfe933af92fa2d4c0cbd5f7e89d595fb"} Jul 07 15:34:44.274233 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:44.274158 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj" event={"ID":"661f6abc-0189-462a-bb21-54dbc46bbf97","Type":"ContainerDied","Data":"1c0ad995a2da2ed46107c6ae27f67400c0835442cbacaac9ed4c80e1b7c70602"} Jul 07 15:34:44.274233 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:44.274179 2565 scope.go:117] "RemoveContainer" containerID="6e4f727e21a2f0542e8c3773af1c8892dfe933af92fa2d4c0cbd5f7e89d595fb" Jul 07 15:34:44.283539 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:44.283520 2565 scope.go:117] "RemoveContainer" containerID="6e4f727e21a2f0542e8c3773af1c8892dfe933af92fa2d4c0cbd5f7e89d595fb" Jul 07 15:34:44.283810 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:34:44.283784 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e4f727e21a2f0542e8c3773af1c8892dfe933af92fa2d4c0cbd5f7e89d595fb\": container with ID starting with 6e4f727e21a2f0542e8c3773af1c8892dfe933af92fa2d4c0cbd5f7e89d595fb not found: ID does not exist" containerID="6e4f727e21a2f0542e8c3773af1c8892dfe933af92fa2d4c0cbd5f7e89d595fb" Jul 07 15:34:44.283862 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:44.283819 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4f727e21a2f0542e8c3773af1c8892dfe933af92fa2d4c0cbd5f7e89d595fb"} err="failed to get container status \"6e4f727e21a2f0542e8c3773af1c8892dfe933af92fa2d4c0cbd5f7e89d595fb\": rpc error: code = NotFound desc = could not find container \"6e4f727e21a2f0542e8c3773af1c8892dfe933af92fa2d4c0cbd5f7e89d595fb\": container with ID starting with 6e4f727e21a2f0542e8c3773af1c8892dfe933af92fa2d4c0cbd5f7e89d595fb not found: ID does not exist" Jul 07 15:34:44.299461 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:44.299435 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj"] Jul 07 15:34:44.304759 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:44.304736 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-8d3dc6-66cb759cc6-b7hpj"] Jul 07 15:34:45.103533 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:45.103499 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="661f6abc-0189-462a-bb21-54dbc46bbf97" path="/var/lib/kubelet/pods/661f6abc-0189-462a-bb21-54dbc46bbf97/volumes" Jul 07 15:34:50.038772 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:50.038737 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx"] Jul 07 15:34:50.039168 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:50.039008 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" podUID="fb251480-7c9e-4e8e-be76-29653cd875ed" containerName="istio-proxy" containerID="cri-o://5bbc69014d4f723e44288b7cef4da7b9894e31becb7b4f5c27b9a9341c18d185" gracePeriod=30 Jul 07 15:34:53.726714 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.726681 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p"] Jul 07 15:34:53.727123 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.727078 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="661f6abc-0189-462a-bb21-54dbc46bbf97" containerName="maas-api" Jul 07 15:34:53.727123 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.727091 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="661f6abc-0189-462a-bb21-54dbc46bbf97" containerName="maas-api" Jul 07 15:34:53.727196 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.727156 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="661f6abc-0189-462a-bb21-54dbc46bbf97" containerName="maas-api" Jul 07 15:34:53.730788 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.730769 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:34:53.733857 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.733836 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-l6lq8\"" Jul 07 15:34:53.733960 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.733914 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-8f5391-serving-cert\"" Jul 07 15:34:53.743626 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.743601 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p"] Jul 07 15:34:53.845541 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.845508 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-maas-api-tls\") pod \"maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p\" (UID: \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\") " pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:34:53.845714 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.845567 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p\" (UID: \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\") " pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:34:53.845714 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.845672 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqdfs\" (UniqueName: \"kubernetes.io/projected/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-kube-api-access-hqdfs\") pod \"maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p\" (UID: \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\") " pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:34:53.947105 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.947064 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqdfs\" (UniqueName: \"kubernetes.io/projected/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-kube-api-access-hqdfs\") pod \"maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p\" (UID: \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\") " pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:34:53.947301 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.947167 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-maas-api-tls\") pod \"maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p\" (UID: \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\") " pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:34:53.947301 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.947227 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p\" (UID: \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\") " pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:34:53.947419 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:34:53.947306 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-8f5391-serving-cert: secret "maas-api-e2e-shared-a-8f5391-serving-cert" not found Jul 07 15:34:53.947419 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:34:53.947405 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-maas-api-tls podName:17d0d7a1-92c7-4f51-90c3-ccb19c0591c0 nodeName:}" failed. No retries permitted until 2026-07-07 15:34:54.447388586 +0000 UTC m=+2023.872776952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-maas-api-tls") pod "maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" (UID: "17d0d7a1-92c7-4f51-90c3-ccb19c0591c0") : secret "maas-api-e2e-shared-a-8f5391-serving-cert" not found Jul 07 15:34:53.947879 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.947861 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p\" (UID: \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\") " pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:34:53.958871 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:53.958847 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqdfs\" (UniqueName: \"kubernetes.io/projected/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-kube-api-access-hqdfs\") pod \"maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p\" (UID: \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\") " pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:34:54.451902 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:54.451866 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-maas-api-tls\") pod \"maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p\" (UID: \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\") " pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:34:54.454544 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:54.454509 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-maas-api-tls\") pod \"maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p\" (UID: \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\") " pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:34:54.645005 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:54.644961 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:34:54.778161 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:54.778136 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p"] Jul 07 15:34:54.779849 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:34:54.779823 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17d0d7a1_92c7_4f51_90c3_ccb19c0591c0.slice/crio-4602da26f2c37814a5a716911b1673c237b3fc9c69ab38a87c951b503b2a019b WatchSource:0}: Error finding container 4602da26f2c37814a5a716911b1673c237b3fc9c69ab38a87c951b503b2a019b: Status 404 returned error can't find the container with id 4602da26f2c37814a5a716911b1673c237b3fc9c69ab38a87c951b503b2a019b Jul 07 15:34:55.285750 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.285728 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:55.338933 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.338899 2565 generic.go:356] "Generic (PLEG): container finished" podID="fb251480-7c9e-4e8e-be76-29653cd875ed" containerID="5bbc69014d4f723e44288b7cef4da7b9894e31becb7b4f5c27b9a9341c18d185" exitCode=0 Jul 07 15:34:55.339067 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.338971 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" event={"ID":"fb251480-7c9e-4e8e-be76-29653cd875ed","Type":"ContainerDied","Data":"5bbc69014d4f723e44288b7cef4da7b9894e31becb7b4f5c27b9a9341c18d185"} Jul 07 15:34:55.339067 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.338988 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" Jul 07 15:34:55.339067 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.339027 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx" event={"ID":"fb251480-7c9e-4e8e-be76-29653cd875ed","Type":"ContainerDied","Data":"b2be22f21ff4a5af4c14f6de909de4164b24f9dd6fc8b72fb086de83bba50ec0"} Jul 07 15:34:55.339067 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.339055 2565 scope.go:117] "RemoveContainer" containerID="5bbc69014d4f723e44288b7cef4da7b9894e31becb7b4f5c27b9a9341c18d185" Jul 07 15:34:55.340378 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.340354 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" event={"ID":"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0","Type":"ContainerStarted","Data":"4602da26f2c37814a5a716911b1673c237b3fc9c69ab38a87c951b503b2a019b"} Jul 07 15:34:55.360783 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.360765 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-envoy\") pod \"fb251480-7c9e-4e8e-be76-29653cd875ed\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " Jul 07 15:34:55.360900 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.360803 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sskxc\" (UniqueName: \"kubernetes.io/projected/fb251480-7c9e-4e8e-be76-29653cd875ed-kube-api-access-sskxc\") pod \"fb251480-7c9e-4e8e-be76-29653cd875ed\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " Jul 07 15:34:55.360900 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.360838 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-workload-socket\") pod \"fb251480-7c9e-4e8e-be76-29653cd875ed\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " Jul 07 15:34:55.361045 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.360963 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-token\") pod \"fb251480-7c9e-4e8e-be76-29653cd875ed\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " Jul 07 15:34:55.361045 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.361028 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb251480-7c9e-4e8e-be76-29653cd875ed-istiod-ca-cert\") pod \"fb251480-7c9e-4e8e-be76-29653cd875ed\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " Jul 07 15:34:55.361168 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.361065 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-podinfo\") pod \"fb251480-7c9e-4e8e-be76-29653cd875ed\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " Jul 07 15:34:55.361168 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.361094 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-credential-socket\") pod \"fb251480-7c9e-4e8e-be76-29653cd875ed\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " Jul 07 15:34:55.361168 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.361132 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-data\") pod \"fb251480-7c9e-4e8e-be76-29653cd875ed\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " Jul 07 15:34:55.361168 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.361163 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-workload-certs\") pod \"fb251480-7c9e-4e8e-be76-29653cd875ed\" (UID: \"fb251480-7c9e-4e8e-be76-29653cd875ed\") " Jul 07 15:34:55.361388 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.361162 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fb251480-7c9e-4e8e-be76-29653cd875ed" (UID: "fb251480-7c9e-4e8e-be76-29653cd875ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:55.361454 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.361423 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fb251480-7c9e-4e8e-be76-29653cd875ed" (UID: "fb251480-7c9e-4e8e-be76-29653cd875ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:55.361892 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.361552 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-workload-socket\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:55.361892 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.361576 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-workload-certs\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:55.361892 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.361638 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fb251480-7c9e-4e8e-be76-29653cd875ed" (UID: "fb251480-7c9e-4e8e-be76-29653cd875ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:55.361892 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.361855 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb251480-7c9e-4e8e-be76-29653cd875ed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fb251480-7c9e-4e8e-be76-29653cd875ed" (UID: "fb251480-7c9e-4e8e-be76-29653cd875ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:34:55.362543 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.362481 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-data" (OuterVolumeSpecName: "istio-data") pod "fb251480-7c9e-4e8e-be76-29653cd875ed" (UID: "fb251480-7c9e-4e8e-be76-29653cd875ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:55.362750 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.362672 2565 scope.go:117] "RemoveContainer" containerID="95f3267d03659c374c0f740f168cbf350247ea442174a35757cc20a8e268a543" Jul 07 15:34:55.363255 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.363226 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fb251480-7c9e-4e8e-be76-29653cd875ed" (UID: "fb251480-7c9e-4e8e-be76-29653cd875ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:55.363680 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.363659 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-token" (OuterVolumeSpecName: "istio-token") pod "fb251480-7c9e-4e8e-be76-29653cd875ed" (UID: "fb251480-7c9e-4e8e-be76-29653cd875ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:34:55.363789 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.363657 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fb251480-7c9e-4e8e-be76-29653cd875ed" (UID: "fb251480-7c9e-4e8e-be76-29653cd875ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:34:55.364548 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.364529 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb251480-7c9e-4e8e-be76-29653cd875ed-kube-api-access-sskxc" (OuterVolumeSpecName: "kube-api-access-sskxc") pod "fb251480-7c9e-4e8e-be76-29653cd875ed" (UID: "fb251480-7c9e-4e8e-be76-29653cd875ed"). InnerVolumeSpecName "kube-api-access-sskxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:34:55.376925 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.376907 2565 scope.go:117] "RemoveContainer" containerID="5bbc69014d4f723e44288b7cef4da7b9894e31becb7b4f5c27b9a9341c18d185" Jul 07 15:34:55.377186 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:34:55.377166 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bbc69014d4f723e44288b7cef4da7b9894e31becb7b4f5c27b9a9341c18d185\": container with ID starting with 5bbc69014d4f723e44288b7cef4da7b9894e31becb7b4f5c27b9a9341c18d185 not found: ID does not exist" containerID="5bbc69014d4f723e44288b7cef4da7b9894e31becb7b4f5c27b9a9341c18d185" Jul 07 15:34:55.377236 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.377196 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bbc69014d4f723e44288b7cef4da7b9894e31becb7b4f5c27b9a9341c18d185"} err="failed to get container status \"5bbc69014d4f723e44288b7cef4da7b9894e31becb7b4f5c27b9a9341c18d185\": rpc error: code = NotFound desc = could not find container \"5bbc69014d4f723e44288b7cef4da7b9894e31becb7b4f5c27b9a9341c18d185\": container with ID starting with 5bbc69014d4f723e44288b7cef4da7b9894e31becb7b4f5c27b9a9341c18d185 not found: ID does not exist" Jul 07 15:34:55.377236 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.377212 2565 scope.go:117] "RemoveContainer" containerID="95f3267d03659c374c0f740f168cbf350247ea442174a35757cc20a8e268a543" Jul 07 15:34:55.377459 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:34:55.377445 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95f3267d03659c374c0f740f168cbf350247ea442174a35757cc20a8e268a543\": container with ID starting with 95f3267d03659c374c0f740f168cbf350247ea442174a35757cc20a8e268a543 not found: ID does not exist" containerID="95f3267d03659c374c0f740f168cbf350247ea442174a35757cc20a8e268a543" Jul 07 15:34:55.377502 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.377463 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95f3267d03659c374c0f740f168cbf350247ea442174a35757cc20a8e268a543"} err="failed to get container status \"95f3267d03659c374c0f740f168cbf350247ea442174a35757cc20a8e268a543\": rpc error: code = NotFound desc = could not find container \"95f3267d03659c374c0f740f168cbf350247ea442174a35757cc20a8e268a543\": container with ID starting with 95f3267d03659c374c0f740f168cbf350247ea442174a35757cc20a8e268a543 not found: ID does not exist" Jul 07 15:34:55.462571 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.462539 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-token\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:55.462571 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.462570 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb251480-7c9e-4e8e-be76-29653cd875ed-istiod-ca-cert\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:55.462571 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.462581 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-podinfo\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:55.462803 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.462589 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-credential-socket\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:55.462803 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.462599 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-data\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:55.462803 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.462608 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb251480-7c9e-4e8e-be76-29653cd875ed-istio-envoy\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:55.462803 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.462618 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sskxc\" (UniqueName: \"kubernetes.io/projected/fb251480-7c9e-4e8e-be76-29653cd875ed-kube-api-access-sskxc\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:34:55.663787 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.663757 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx"] Jul 07 15:34:55.668899 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:55.668870 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-8d3dc6-openshift-default-5596d7b4d5-k46zx"] Jul 07 15:34:56.348210 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:56.348175 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" event={"ID":"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0","Type":"ContainerStarted","Data":"bdf5e5294f569de4c81c49da027bd0b9bbf19ed5cfc8c689f20401798db1507b"} Jul 07 15:34:56.348583 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:56.348276 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:34:56.370013 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:56.369969 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" podStartSLOduration=2.830607796 podStartE2EDuration="3.369953456s" totalImagesPullingTime="539.34566ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:34:56.36832743 +0000 UTC m=+2025.793715818" watchObservedRunningTime="2026-07-07 15:34:56.369953456 +0000 UTC m=+2025.795341838" Jul 07 15:34:57.104718 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:34:57.104684 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb251480-7c9e-4e8e-be76-29653cd875ed" path="/var/lib/kubelet/pods/fb251480-7c9e-4e8e-be76-29653cd875ed/volumes" Jul 07 15:35:01.982328 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:01.982290 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456"] Jul 07 15:35:01.982814 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:01.982659 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb251480-7c9e-4e8e-be76-29653cd875ed" containerName="istio-proxy" Jul 07 15:35:01.982814 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:01.982674 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb251480-7c9e-4e8e-be76-29653cd875ed" containerName="istio-proxy" Jul 07 15:35:01.982814 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:01.982689 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb251480-7c9e-4e8e-be76-29653cd875ed" containerName="istio-proxy" Jul 07 15:35:01.982814 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:01.982696 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb251480-7c9e-4e8e-be76-29653cd875ed" containerName="istio-proxy" Jul 07 15:35:01.982814 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:01.982790 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb251480-7c9e-4e8e-be76-29653cd875ed" containerName="istio-proxy" Jul 07 15:35:01.987019 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:01.986993 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:35:01.989915 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:01.989892 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-600c1d-serving-cert\"" Jul 07 15:35:02.003689 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.003663 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456"] Jul 07 15:35:02.119393 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.119356 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5mcv\" (UniqueName: \"kubernetes.io/projected/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-kube-api-access-h5mcv\") pod \"maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456\" (UID: \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\") " pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:35:02.119393 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.119395 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-maas-api-tls\") pod \"maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456\" (UID: \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\") " pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:35:02.119598 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.119475 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-openshift-service-ca\") pod \"maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456\" (UID: \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\") " pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:35:02.220432 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.220389 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5mcv\" (UniqueName: \"kubernetes.io/projected/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-kube-api-access-h5mcv\") pod \"maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456\" (UID: \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\") " pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:35:02.220604 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.220450 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-maas-api-tls\") pod \"maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456\" (UID: \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\") " pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:35:02.220604 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.220506 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-openshift-service-ca\") pod \"maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456\" (UID: \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\") " pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:35:02.221125 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.221091 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-openshift-service-ca\") pod \"maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456\" (UID: \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\") " pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:35:02.223250 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.223224 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-maas-api-tls\") pod \"maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456\" (UID: \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\") " pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:35:02.229664 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.229635 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5mcv\" (UniqueName: \"kubernetes.io/projected/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-kube-api-access-h5mcv\") pod \"maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456\" (UID: \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\") " pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:35:02.297646 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.297563 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:35:02.362243 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.362214 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:35:02.467432 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.467393 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456"] Jul 07 15:35:02.468358 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:35:02.468334 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb33cdbcb_c13c_4fbe_a9a1_c409b965a486.slice/crio-22486eef686b6d60f2f571633b8dc685fde0739fb4b631ba90cfda6027d8e958 WatchSource:0}: Error finding container 22486eef686b6d60f2f571633b8dc685fde0739fb4b631ba90cfda6027d8e958: Status 404 returned error can't find the container with id 22486eef686b6d60f2f571633b8dc685fde0739fb4b631ba90cfda6027d8e958 Jul 07 15:35:02.469760 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:02.469742 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:35:03.393472 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:03.393432 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" event={"ID":"b33cdbcb-c13c-4fbe-a9a1-c409b965a486","Type":"ContainerStarted","Data":"279c53fd6798eca9a98ebad422c806509ce74225a510e929b1e86fc2968c7ed2"} Jul 07 15:35:03.393918 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:03.393513 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" event={"ID":"b33cdbcb-c13c-4fbe-a9a1-c409b965a486","Type":"ContainerStarted","Data":"22486eef686b6d60f2f571633b8dc685fde0739fb4b631ba90cfda6027d8e958"} Jul 07 15:35:03.393918 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:03.393591 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:35:03.415167 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:03.415115 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" podStartSLOduration=2.109996543 podStartE2EDuration="2.415096693s" totalImagesPullingTime="305.10015ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:35:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:35:03.413141019 +0000 UTC m=+2032.838529411" watchObservedRunningTime="2026-07-07 15:35:03.415096693 +0000 UTC m=+2032.840485082" Jul 07 15:35:09.405442 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:35:09.405416 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:36:11.172726 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:36:11.172690 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:36:11.183185 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:36:11.183161 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:38:18.579285 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.579253 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x"] Jul 07 15:38:18.579863 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.579803 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb251480-7c9e-4e8e-be76-29653cd875ed" containerName="istio-proxy" Jul 07 15:38:18.582888 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.582872 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.585370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.585341 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8f5391\"/\"kube-root-ca.crt\"" Jul 07 15:38:18.585484 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.585383 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8f5391\"/\"default-dockercfg-k9f9m\"" Jul 07 15:38:18.586354 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.586328 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8f5391\"/\"sub-test-model-8f5391-kserve-self-signed-certs\"" Jul 07 15:38:18.586354 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.586329 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8f5391\"/\"openshift-service-ca.crt\"" Jul 07 15:38:18.592778 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.592755 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x"] Jul 07 15:38:18.734154 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.734114 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-home\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.734352 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.734166 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbd5r\" (UniqueName: \"kubernetes.io/projected/cabce5b6-968c-498e-8c46-7346a6c77a09-kube-api-access-vbd5r\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.734352 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.734229 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-dshm\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.734352 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.734280 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-model-cache\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.734352 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.734306 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-tmp-dir\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.734529 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.734361 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cabce5b6-968c-498e-8c46-7346a6c77a09-tls-certs\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.835671 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.835583 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-dshm\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.835671 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.835637 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-model-cache\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.835671 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.835656 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-tmp-dir\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.835988 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.835680 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cabce5b6-968c-498e-8c46-7346a6c77a09-tls-certs\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.835988 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.835725 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-home\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.835988 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.835758 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbd5r\" (UniqueName: \"kubernetes.io/projected/cabce5b6-968c-498e-8c46-7346a6c77a09-kube-api-access-vbd5r\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.836162 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.836126 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-tmp-dir\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.836244 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.836167 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-model-cache\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.836244 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.836209 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-home\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.837976 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.837928 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-dshm\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.838313 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.838294 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cabce5b6-968c-498e-8c46-7346a6c77a09-tls-certs\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.847053 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.847025 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbd5r\" (UniqueName: \"kubernetes.io/projected/cabce5b6-968c-498e-8c46-7346a6c77a09-kube-api-access-vbd5r\") pod \"sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:18.893033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:18.893007 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:19.026372 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:19.026347 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x"] Jul 07 15:38:19.027689 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:38:19.027663 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcabce5b6_968c_498e_8c46_7346a6c77a09.slice/crio-d2c98a436630a51426c39e5831b186e77e74c28e484bbb6b68111533e5ca418e WatchSource:0}: Error finding container d2c98a436630a51426c39e5831b186e77e74c28e484bbb6b68111533e5ca418e: Status 404 returned error can't find the container with id d2c98a436630a51426c39e5831b186e77e74c28e484bbb6b68111533e5ca418e Jul 07 15:38:19.429962 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:19.429899 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" event={"ID":"cabce5b6-968c-498e-8c46-7346a6c77a09","Type":"ContainerStarted","Data":"ea7f709a4ccbc7a558139d96d1a96e4b75b7104d6b4d3914c729c74253d52893"} Jul 07 15:38:19.430170 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:19.429972 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" event={"ID":"cabce5b6-968c-498e-8c46-7346a6c77a09","Type":"ContainerStarted","Data":"d2c98a436630a51426c39e5831b186e77e74c28e484bbb6b68111533e5ca418e"} Jul 07 15:38:19.450695 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:19.450644 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" podStartSLOduration=1.450629271 podStartE2EDuration="1.450629271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:38:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:38:19.448701005 +0000 UTC m=+2228.874089417" watchObservedRunningTime="2026-07-07 15:38:19.450629271 +0000 UTC m=+2228.876017659" Jul 07 15:38:28.893876 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:28.893840 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:28.894365 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:28.894128 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:28.906458 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:28.906430 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:38:29.492508 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:38:29.492477 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:39:16.715898 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:16.715863 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x"] Jul 07 15:39:31.735586 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:31.735526 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" podUID="cabce5b6-968c-498e-8c46-7346a6c77a09" containerName="main" containerID="cri-o://ea7f709a4ccbc7a558139d96d1a96e4b75b7104d6b4d3914c729c74253d52893" gracePeriod=45 Jul 07 15:39:31.998984 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:31.998911 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:39:32.087433 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.087393 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-tmp-dir\") pod \"cabce5b6-968c-498e-8c46-7346a6c77a09\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " Jul 07 15:39:32.087606 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.087456 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-dshm\") pod \"cabce5b6-968c-498e-8c46-7346a6c77a09\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " Jul 07 15:39:32.087606 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.087512 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-model-cache\") pod \"cabce5b6-968c-498e-8c46-7346a6c77a09\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " Jul 07 15:39:32.087606 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.087561 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbd5r\" (UniqueName: \"kubernetes.io/projected/cabce5b6-968c-498e-8c46-7346a6c77a09-kube-api-access-vbd5r\") pod \"cabce5b6-968c-498e-8c46-7346a6c77a09\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " Jul 07 15:39:32.087606 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.087580 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cabce5b6-968c-498e-8c46-7346a6c77a09-tls-certs\") pod \"cabce5b6-968c-498e-8c46-7346a6c77a09\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " Jul 07 15:39:32.087606 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.087601 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-home\") pod \"cabce5b6-968c-498e-8c46-7346a6c77a09\" (UID: \"cabce5b6-968c-498e-8c46-7346a6c77a09\") " Jul 07 15:39:32.087861 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.087693 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cabce5b6-968c-498e-8c46-7346a6c77a09" (UID: "cabce5b6-968c-498e-8c46-7346a6c77a09"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:39:32.087918 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.087861 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-tmp-dir\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:39:32.087918 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.087897 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-home" (OuterVolumeSpecName: "home") pod "cabce5b6-968c-498e-8c46-7346a6c77a09" (UID: "cabce5b6-968c-498e-8c46-7346a6c77a09"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:39:32.088041 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.087913 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-model-cache" (OuterVolumeSpecName: "model-cache") pod "cabce5b6-968c-498e-8c46-7346a6c77a09" (UID: "cabce5b6-968c-498e-8c46-7346a6c77a09"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:39:32.089733 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.089707 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-dshm" (OuterVolumeSpecName: "dshm") pod "cabce5b6-968c-498e-8c46-7346a6c77a09" (UID: "cabce5b6-968c-498e-8c46-7346a6c77a09"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:39:32.089846 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.089709 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cabce5b6-968c-498e-8c46-7346a6c77a09-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cabce5b6-968c-498e-8c46-7346a6c77a09" (UID: "cabce5b6-968c-498e-8c46-7346a6c77a09"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:39:32.089846 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.089756 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cabce5b6-968c-498e-8c46-7346a6c77a09-kube-api-access-vbd5r" (OuterVolumeSpecName: "kube-api-access-vbd5r") pod "cabce5b6-968c-498e-8c46-7346a6c77a09" (UID: "cabce5b6-968c-498e-8c46-7346a6c77a09"). InnerVolumeSpecName "kube-api-access-vbd5r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:39:32.188763 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.188717 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-dshm\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:39:32.188763 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.188746 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-model-cache\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:39:32.188763 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.188761 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbd5r\" (UniqueName: \"kubernetes.io/projected/cabce5b6-968c-498e-8c46-7346a6c77a09-kube-api-access-vbd5r\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:39:32.188763 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.188775 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cabce5b6-968c-498e-8c46-7346a6c77a09-tls-certs\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:39:32.189076 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.188787 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cabce5b6-968c-498e-8c46-7346a6c77a09-home\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:39:32.822670 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.822631 2565 generic.go:356] "Generic (PLEG): container finished" podID="cabce5b6-968c-498e-8c46-7346a6c77a09" containerID="ea7f709a4ccbc7a558139d96d1a96e4b75b7104d6b4d3914c729c74253d52893" exitCode=0 Jul 07 15:39:32.823116 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.822682 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" Jul 07 15:39:32.823116 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.822716 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" event={"ID":"cabce5b6-968c-498e-8c46-7346a6c77a09","Type":"ContainerDied","Data":"ea7f709a4ccbc7a558139d96d1a96e4b75b7104d6b4d3914c729c74253d52893"} Jul 07 15:39:32.823116 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.822760 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x" event={"ID":"cabce5b6-968c-498e-8c46-7346a6c77a09","Type":"ContainerDied","Data":"d2c98a436630a51426c39e5831b186e77e74c28e484bbb6b68111533e5ca418e"} Jul 07 15:39:32.823116 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.822781 2565 scope.go:117] "RemoveContainer" containerID="ea7f709a4ccbc7a558139d96d1a96e4b75b7104d6b4d3914c729c74253d52893" Jul 07 15:39:32.831994 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.831975 2565 scope.go:117] "RemoveContainer" containerID="ea7f709a4ccbc7a558139d96d1a96e4b75b7104d6b4d3914c729c74253d52893" Jul 07 15:39:32.832263 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:39:32.832243 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea7f709a4ccbc7a558139d96d1a96e4b75b7104d6b4d3914c729c74253d52893\": container with ID starting with ea7f709a4ccbc7a558139d96d1a96e4b75b7104d6b4d3914c729c74253d52893 not found: ID does not exist" containerID="ea7f709a4ccbc7a558139d96d1a96e4b75b7104d6b4d3914c729c74253d52893" Jul 07 15:39:32.832322 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.832272 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea7f709a4ccbc7a558139d96d1a96e4b75b7104d6b4d3914c729c74253d52893"} err="failed to get container status \"ea7f709a4ccbc7a558139d96d1a96e4b75b7104d6b4d3914c729c74253d52893\": rpc error: code = NotFound desc = could not find container \"ea7f709a4ccbc7a558139d96d1a96e4b75b7104d6b4d3914c729c74253d52893\": container with ID starting with ea7f709a4ccbc7a558139d96d1a96e4b75b7104d6b4d3914c729c74253d52893 not found: ID does not exist" Jul 07 15:39:32.847229 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.847193 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x"] Jul 07 15:39:32.852507 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:32.852483 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8f5391/sub-test-model-8f5391-kserve-7c5589d4d7-jjt9x"] Jul 07 15:39:33.103740 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:33.103661 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cabce5b6-968c-498e-8c46-7346a6c77a09" path="/var/lib/kubelet/pods/cabce5b6-968c-498e-8c46-7346a6c77a09/volumes" Jul 07 15:39:55.683213 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.683173 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb"] Jul 07 15:39:55.685709 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.683600 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cabce5b6-968c-498e-8c46-7346a6c77a09" containerName="main" Jul 07 15:39:55.685709 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.683614 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cabce5b6-968c-498e-8c46-7346a6c77a09" containerName="main" Jul 07 15:39:55.685709 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.683702 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="cabce5b6-968c-498e-8c46-7346a6c77a09" containerName="main" Jul 07 15:39:55.686566 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.686550 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.690791 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.690767 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-600c1d\"/\"default-dockercfg-nblnj\"" Jul 07 15:39:55.690791 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.690787 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-600c1d\"/\"openshift-service-ca.crt\"" Jul 07 15:39:55.691010 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.690769 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-600c1d\"/\"kube-root-ca.crt\"" Jul 07 15:39:55.691010 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.690769 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-600c1d\"/\"rate-test-model-600c1d-kserve-self-signed-certs\"" Jul 07 15:39:55.700696 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.700671 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb"] Jul 07 15:39:55.796078 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.796041 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef1f01e-209c-43c2-a781-31ea117c8259-tls-certs\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.796078 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.796076 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-model-cache\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.796288 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.796145 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-dshm\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.796288 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.796163 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-home\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.796288 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.796219 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-tmp-dir\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.796288 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.796273 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trkmk\" (UniqueName: \"kubernetes.io/projected/7ef1f01e-209c-43c2-a781-31ea117c8259-kube-api-access-trkmk\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.896818 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.896782 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef1f01e-209c-43c2-a781-31ea117c8259-tls-certs\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.896818 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.896817 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-model-cache\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.897078 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.896873 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-dshm\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.897078 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.896889 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-home\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.897078 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.896905 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-tmp-dir\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.897078 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.896978 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trkmk\" (UniqueName: \"kubernetes.io/projected/7ef1f01e-209c-43c2-a781-31ea117c8259-kube-api-access-trkmk\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.897417 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.897394 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-home\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.897417 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.897410 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-tmp-dir\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.897548 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.897446 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-model-cache\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.899085 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.899060 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-dshm\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.899404 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.899387 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef1f01e-209c-43c2-a781-31ea117c8259-tls-certs\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.906510 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.906488 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trkmk\" (UniqueName: \"kubernetes.io/projected/7ef1f01e-209c-43c2-a781-31ea117c8259-kube-api-access-trkmk\") pod \"rate-test-model-600c1d-kserve-fc466dd8c-2f8zb\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:55.996063 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:55.995984 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:39:56.132113 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:56.132082 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb"] Jul 07 15:39:56.133204 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:39:56.133182 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ef1f01e_209c_43c2_a781_31ea117c8259.slice/crio-1e09e41498a95bca27668e7ee136b27fc77ed18e013382b39a1c0cd08e6d5380 WatchSource:0}: Error finding container 1e09e41498a95bca27668e7ee136b27fc77ed18e013382b39a1c0cd08e6d5380: Status 404 returned error can't find the container with id 1e09e41498a95bca27668e7ee136b27fc77ed18e013382b39a1c0cd08e6d5380 Jul 07 15:39:56.948575 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:56.948539 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" event={"ID":"7ef1f01e-209c-43c2-a781-31ea117c8259","Type":"ContainerStarted","Data":"2d756d9c66c3803ce4923824c53a0ec32ef4d363769b107f9a1a60b12c5772d4"} Jul 07 15:39:56.949081 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:56.948583 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" event={"ID":"7ef1f01e-209c-43c2-a781-31ea117c8259","Type":"ContainerStarted","Data":"1e09e41498a95bca27668e7ee136b27fc77ed18e013382b39a1c0cd08e6d5380"} Jul 07 15:39:56.979093 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:39:56.979037 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" podStartSLOduration=1.97901664 podStartE2EDuration="1.97901664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:39:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:39:56.978763501 +0000 UTC m=+2326.404151890" watchObservedRunningTime="2026-07-07 15:39:56.97901664 +0000 UTC m=+2326.404405027" Jul 07 15:40:05.996461 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:05.996368 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:40:05.996461 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:05.996447 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:40:06.008927 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:06.008903 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:40:07.011915 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:07.011880 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:40:38.600816 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:38.600785 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb"] Jul 07 15:40:40.573427 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:40.573388 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p"] Jul 07 15:40:40.573865 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:40.573612 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" podUID="17d0d7a1-92c7-4f51-90c3-ccb19c0591c0" containerName="maas-api" containerID="cri-o://bdf5e5294f569de4c81c49da027bd0b9bbf19ed5cfc8c689f20401798db1507b" gracePeriod=30 Jul 07 15:40:40.825222 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:40.825163 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:40:40.904068 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:40.904021 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-maas-api-tls\") pod \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\" (UID: \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\") " Jul 07 15:40:40.904068 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:40.904081 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-openshift-service-ca\") pod \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\" (UID: \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\") " Jul 07 15:40:40.904338 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:40.904126 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqdfs\" (UniqueName: \"kubernetes.io/projected/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-kube-api-access-hqdfs\") pod \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\" (UID: \"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0\") " Jul 07 15:40:40.904457 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:40.904432 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "17d0d7a1-92c7-4f51-90c3-ccb19c0591c0" (UID: "17d0d7a1-92c7-4f51-90c3-ccb19c0591c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:40:40.906335 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:40.906303 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "17d0d7a1-92c7-4f51-90c3-ccb19c0591c0" (UID: "17d0d7a1-92c7-4f51-90c3-ccb19c0591c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:40:40.906335 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:40.906317 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-kube-api-access-hqdfs" (OuterVolumeSpecName: "kube-api-access-hqdfs") pod "17d0d7a1-92c7-4f51-90c3-ccb19c0591c0" (UID: "17d0d7a1-92c7-4f51-90c3-ccb19c0591c0"). InnerVolumeSpecName "kube-api-access-hqdfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:40:41.005131 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:41.005094 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-maas-api-tls\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:40:41.005131 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:41.005126 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-openshift-service-ca\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:40:41.005131 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:41.005136 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqdfs\" (UniqueName: \"kubernetes.io/projected/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0-kube-api-access-hqdfs\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:40:41.186094 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:41.186057 2565 generic.go:356] "Generic (PLEG): container finished" podID="17d0d7a1-92c7-4f51-90c3-ccb19c0591c0" containerID="bdf5e5294f569de4c81c49da027bd0b9bbf19ed5cfc8c689f20401798db1507b" exitCode=0 Jul 07 15:40:41.186263 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:41.186101 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" Jul 07 15:40:41.186263 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:41.186114 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" event={"ID":"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0","Type":"ContainerDied","Data":"bdf5e5294f569de4c81c49da027bd0b9bbf19ed5cfc8c689f20401798db1507b"} Jul 07 15:40:41.186263 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:41.186162 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p" event={"ID":"17d0d7a1-92c7-4f51-90c3-ccb19c0591c0","Type":"ContainerDied","Data":"4602da26f2c37814a5a716911b1673c237b3fc9c69ab38a87c951b503b2a019b"} Jul 07 15:40:41.186263 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:41.186187 2565 scope.go:117] "RemoveContainer" containerID="bdf5e5294f569de4c81c49da027bd0b9bbf19ed5cfc8c689f20401798db1507b" Jul 07 15:40:41.195209 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:41.195193 2565 scope.go:117] "RemoveContainer" containerID="bdf5e5294f569de4c81c49da027bd0b9bbf19ed5cfc8c689f20401798db1507b" Jul 07 15:40:41.195467 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:40:41.195446 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdf5e5294f569de4c81c49da027bd0b9bbf19ed5cfc8c689f20401798db1507b\": container with ID starting with bdf5e5294f569de4c81c49da027bd0b9bbf19ed5cfc8c689f20401798db1507b not found: ID does not exist" containerID="bdf5e5294f569de4c81c49da027bd0b9bbf19ed5cfc8c689f20401798db1507b" Jul 07 15:40:41.195519 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:41.195475 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdf5e5294f569de4c81c49da027bd0b9bbf19ed5cfc8c689f20401798db1507b"} err="failed to get container status \"bdf5e5294f569de4c81c49da027bd0b9bbf19ed5cfc8c689f20401798db1507b\": rpc error: code = NotFound desc = could not find container \"bdf5e5294f569de4c81c49da027bd0b9bbf19ed5cfc8c689f20401798db1507b\": container with ID starting with bdf5e5294f569de4c81c49da027bd0b9bbf19ed5cfc8c689f20401798db1507b not found: ID does not exist" Jul 07 15:40:41.205581 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:41.205558 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p"] Jul 07 15:40:41.208145 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:41.208115 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8f5391-5bdbbcb578-szv7p"] Jul 07 15:40:43.110162 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:43.110126 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17d0d7a1-92c7-4f51-90c3-ccb19c0591c0" path="/var/lib/kubelet/pods/17d0d7a1-92c7-4f51-90c3-ccb19c0591c0/volumes" Jul 07 15:40:53.620925 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:53.620851 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" podUID="7ef1f01e-209c-43c2-a781-31ea117c8259" containerName="main" containerID="cri-o://2d756d9c66c3803ce4923824c53a0ec32ef4d363769b107f9a1a60b12c5772d4" gracePeriod=45 Jul 07 15:40:53.870620 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:53.870595 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:40:54.024308 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.024273 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trkmk\" (UniqueName: \"kubernetes.io/projected/7ef1f01e-209c-43c2-a781-31ea117c8259-kube-api-access-trkmk\") pod \"7ef1f01e-209c-43c2-a781-31ea117c8259\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " Jul 07 15:40:54.024308 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.024316 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-model-cache\") pod \"7ef1f01e-209c-43c2-a781-31ea117c8259\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " Jul 07 15:40:54.024566 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.024356 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-home\") pod \"7ef1f01e-209c-43c2-a781-31ea117c8259\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " Jul 07 15:40:54.024566 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.024373 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-tmp-dir\") pod \"7ef1f01e-209c-43c2-a781-31ea117c8259\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " Jul 07 15:40:54.024566 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.024440 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-dshm\") pod \"7ef1f01e-209c-43c2-a781-31ea117c8259\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " Jul 07 15:40:54.024566 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.024468 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef1f01e-209c-43c2-a781-31ea117c8259-tls-certs\") pod \"7ef1f01e-209c-43c2-a781-31ea117c8259\" (UID: \"7ef1f01e-209c-43c2-a781-31ea117c8259\") " Jul 07 15:40:54.024776 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.024636 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-model-cache" (OuterVolumeSpecName: "model-cache") pod "7ef1f01e-209c-43c2-a781-31ea117c8259" (UID: "7ef1f01e-209c-43c2-a781-31ea117c8259"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:40:54.024776 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.024718 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7ef1f01e-209c-43c2-a781-31ea117c8259" (UID: "7ef1f01e-209c-43c2-a781-31ea117c8259"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:40:54.024776 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.024730 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-home" (OuterVolumeSpecName: "home") pod "7ef1f01e-209c-43c2-a781-31ea117c8259" (UID: "7ef1f01e-209c-43c2-a781-31ea117c8259"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:40:54.025004 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.024816 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-model-cache\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:40:54.025004 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.024834 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-home\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:40:54.025004 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.024848 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-tmp-dir\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:40:54.026633 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.026610 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-dshm" (OuterVolumeSpecName: "dshm") pod "7ef1f01e-209c-43c2-a781-31ea117c8259" (UID: "7ef1f01e-209c-43c2-a781-31ea117c8259"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:40:54.026760 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.026742 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ef1f01e-209c-43c2-a781-31ea117c8259-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7ef1f01e-209c-43c2-a781-31ea117c8259" (UID: "7ef1f01e-209c-43c2-a781-31ea117c8259"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:40:54.026799 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.026753 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ef1f01e-209c-43c2-a781-31ea117c8259-kube-api-access-trkmk" (OuterVolumeSpecName: "kube-api-access-trkmk") pod "7ef1f01e-209c-43c2-a781-31ea117c8259" (UID: "7ef1f01e-209c-43c2-a781-31ea117c8259"). InnerVolumeSpecName "kube-api-access-trkmk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:40:54.125518 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.125485 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ef1f01e-209c-43c2-a781-31ea117c8259-dshm\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:40:54.125518 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.125514 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ef1f01e-209c-43c2-a781-31ea117c8259-tls-certs\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:40:54.125518 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.125526 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trkmk\" (UniqueName: \"kubernetes.io/projected/7ef1f01e-209c-43c2-a781-31ea117c8259-kube-api-access-trkmk\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:40:54.258026 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.257989 2565 generic.go:356] "Generic (PLEG): container finished" podID="7ef1f01e-209c-43c2-a781-31ea117c8259" containerID="2d756d9c66c3803ce4923824c53a0ec32ef4d363769b107f9a1a60b12c5772d4" exitCode=0 Jul 07 15:40:54.258215 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.258032 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" event={"ID":"7ef1f01e-209c-43c2-a781-31ea117c8259","Type":"ContainerDied","Data":"2d756d9c66c3803ce4923824c53a0ec32ef4d363769b107f9a1a60b12c5772d4"} Jul 07 15:40:54.258215 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.258060 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" event={"ID":"7ef1f01e-209c-43c2-a781-31ea117c8259","Type":"ContainerDied","Data":"1e09e41498a95bca27668e7ee136b27fc77ed18e013382b39a1c0cd08e6d5380"} Jul 07 15:40:54.258215 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.258035 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb" Jul 07 15:40:54.258215 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.258079 2565 scope.go:117] "RemoveContainer" containerID="2d756d9c66c3803ce4923824c53a0ec32ef4d363769b107f9a1a60b12c5772d4" Jul 07 15:40:54.267175 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.267159 2565 scope.go:117] "RemoveContainer" containerID="2d756d9c66c3803ce4923824c53a0ec32ef4d363769b107f9a1a60b12c5772d4" Jul 07 15:40:54.267390 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:40:54.267373 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d756d9c66c3803ce4923824c53a0ec32ef4d363769b107f9a1a60b12c5772d4\": container with ID starting with 2d756d9c66c3803ce4923824c53a0ec32ef4d363769b107f9a1a60b12c5772d4 not found: ID does not exist" containerID="2d756d9c66c3803ce4923824c53a0ec32ef4d363769b107f9a1a60b12c5772d4" Jul 07 15:40:54.267446 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.267397 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d756d9c66c3803ce4923824c53a0ec32ef4d363769b107f9a1a60b12c5772d4"} err="failed to get container status \"2d756d9c66c3803ce4923824c53a0ec32ef4d363769b107f9a1a60b12c5772d4\": rpc error: code = NotFound desc = could not find container \"2d756d9c66c3803ce4923824c53a0ec32ef4d363769b107f9a1a60b12c5772d4\": container with ID starting with 2d756d9c66c3803ce4923824c53a0ec32ef4d363769b107f9a1a60b12c5772d4 not found: ID does not exist" Jul 07 15:40:54.284891 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.284828 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb"] Jul 07 15:40:54.286816 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:54.286798 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-600c1d/rate-test-model-600c1d-kserve-fc466dd8c-2f8zb"] Jul 07 15:40:55.104821 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:40:55.104789 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ef1f01e-209c-43c2-a781-31ea117c8259" path="/var/lib/kubelet/pods/7ef1f01e-209c-43c2-a781-31ea117c8259/volumes" Jul 07 15:41:11.206281 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:11.206249 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:41:11.217115 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:11.217092 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snw9m_2def56c2-e1cd-4af7-8d6b-00e56dc39e1e/ovn-acl-logging/0.log" Jul 07 15:41:24.136803 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.136766 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456"] Jul 07 15:41:24.137276 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.137037 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" podUID="b33cdbcb-c13c-4fbe-a9a1-c409b965a486" containerName="maas-api" containerID="cri-o://279c53fd6798eca9a98ebad422c806509ce74225a510e929b1e86fc2968c7ed2" gracePeriod=30 Jul 07 15:41:24.384624 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.384593 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:41:24.417153 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.417125 2565 generic.go:356] "Generic (PLEG): container finished" podID="b33cdbcb-c13c-4fbe-a9a1-c409b965a486" containerID="279c53fd6798eca9a98ebad422c806509ce74225a510e929b1e86fc2968c7ed2" exitCode=0 Jul 07 15:41:24.417305 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.417169 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" Jul 07 15:41:24.417305 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.417213 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" event={"ID":"b33cdbcb-c13c-4fbe-a9a1-c409b965a486","Type":"ContainerDied","Data":"279c53fd6798eca9a98ebad422c806509ce74225a510e929b1e86fc2968c7ed2"} Jul 07 15:41:24.417394 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.417301 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456" event={"ID":"b33cdbcb-c13c-4fbe-a9a1-c409b965a486","Type":"ContainerDied","Data":"22486eef686b6d60f2f571633b8dc685fde0739fb4b631ba90cfda6027d8e958"} Jul 07 15:41:24.417394 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.417335 2565 scope.go:117] "RemoveContainer" containerID="279c53fd6798eca9a98ebad422c806509ce74225a510e929b1e86fc2968c7ed2" Jul 07 15:41:24.426088 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.426065 2565 scope.go:117] "RemoveContainer" containerID="279c53fd6798eca9a98ebad422c806509ce74225a510e929b1e86fc2968c7ed2" Jul 07 15:41:24.426338 ip-10-0-141-205 kubenswrapper[2565]: E0707 15:41:24.426312 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"279c53fd6798eca9a98ebad422c806509ce74225a510e929b1e86fc2968c7ed2\": container with ID starting with 279c53fd6798eca9a98ebad422c806509ce74225a510e929b1e86fc2968c7ed2 not found: ID does not exist" containerID="279c53fd6798eca9a98ebad422c806509ce74225a510e929b1e86fc2968c7ed2" Jul 07 15:41:24.426390 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.426352 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279c53fd6798eca9a98ebad422c806509ce74225a510e929b1e86fc2968c7ed2"} err="failed to get container status \"279c53fd6798eca9a98ebad422c806509ce74225a510e929b1e86fc2968c7ed2\": rpc error: code = NotFound desc = could not find container \"279c53fd6798eca9a98ebad422c806509ce74225a510e929b1e86fc2968c7ed2\": container with ID starting with 279c53fd6798eca9a98ebad422c806509ce74225a510e929b1e86fc2968c7ed2 not found: ID does not exist" Jul 07 15:41:24.479350 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.479315 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-openshift-service-ca\") pod \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\" (UID: \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\") " Jul 07 15:41:24.479530 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.479355 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5mcv\" (UniqueName: \"kubernetes.io/projected/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-kube-api-access-h5mcv\") pod \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\" (UID: \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\") " Jul 07 15:41:24.479530 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.479439 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-maas-api-tls\") pod \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\" (UID: \"b33cdbcb-c13c-4fbe-a9a1-c409b965a486\") " Jul 07 15:41:24.479709 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.479687 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b33cdbcb-c13c-4fbe-a9a1-c409b965a486" (UID: "b33cdbcb-c13c-4fbe-a9a1-c409b965a486"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:41:24.481447 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.481418 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b33cdbcb-c13c-4fbe-a9a1-c409b965a486" (UID: "b33cdbcb-c13c-4fbe-a9a1-c409b965a486"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:41:24.481553 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.481529 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-kube-api-access-h5mcv" (OuterVolumeSpecName: "kube-api-access-h5mcv") pod "b33cdbcb-c13c-4fbe-a9a1-c409b965a486" (UID: "b33cdbcb-c13c-4fbe-a9a1-c409b965a486"). InnerVolumeSpecName "kube-api-access-h5mcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:41:24.580307 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.580271 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-openshift-service-ca\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:41:24.580307 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.580298 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5mcv\" (UniqueName: \"kubernetes.io/projected/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-kube-api-access-h5mcv\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:41:24.580307 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.580309 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b33cdbcb-c13c-4fbe-a9a1-c409b965a486-maas-api-tls\") on node \"ip-10-0-141-205.ec2.internal\" DevicePath \"\"" Jul 07 15:41:24.742339 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.742292 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456"] Jul 07 15:41:24.744343 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:24.744323 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-600c1d-5cc4d45f5c-92456"] Jul 07 15:41:25.104713 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:25.104628 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b33cdbcb-c13c-4fbe-a9a1-c409b965a486" path="/var/lib/kubelet/pods/b33cdbcb-c13c-4fbe-a9a1-c409b965a486/volumes" Jul 07 15:41:37.000869 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:37.000794 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723970-d5bxv_9526c65e-7436-41dc-8a8b-afe7a07b0dad/cleanup/0.log" Jul 07 15:41:37.519394 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:37.519360 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845dc9964f-2w2h8_10a2db9b-1d35-4dfc-af70-cbf80dd72ff6/copy-manifests/0.log" Jul 07 15:41:37.527494 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:37.527473 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845dc9964f-2w2h8_10a2db9b-1d35-4dfc-af70-cbf80dd72ff6/manager/0.log" Jul 07 15:41:38.721298 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:38.721270 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb_8a99e0df-82c6-4f71-b61c-41c8069311d2/util/0.log" Jul 07 15:41:38.730216 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:38.730176 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb_8a99e0df-82c6-4f71-b61c-41c8069311d2/pull/0.log" Jul 07 15:41:38.739084 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:38.739059 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb_8a99e0df-82c6-4f71-b61c-41c8069311d2/extract/0.log" Jul 07 15:41:38.861178 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:38.861148 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh_fb5faf61-8542-4065-a260-31669e2af05f/util/0.log" Jul 07 15:41:38.869175 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:38.869154 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh_fb5faf61-8542-4065-a260-31669e2af05f/pull/0.log" Jul 07 15:41:38.877026 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:38.877006 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh_fb5faf61-8542-4065-a260-31669e2af05f/extract/0.log" Jul 07 15:41:38.996259 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:38.996174 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw_f5cf3c2a-1a83-407e-bcad-a03da4e78d1c/util/0.log" Jul 07 15:41:39.003040 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:39.003022 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw_f5cf3c2a-1a83-407e-bcad-a03da4e78d1c/pull/0.log" Jul 07 15:41:39.009979 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:39.009933 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw_f5cf3c2a-1a83-407e-bcad-a03da4e78d1c/extract/0.log" Jul 07 15:41:39.125264 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:39.125237 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9_e1db2e0e-d5ae-4949-83dd-8e2b07d50247/util/0.log" Jul 07 15:41:39.134423 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:39.134387 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9_e1db2e0e-d5ae-4949-83dd-8e2b07d50247/pull/0.log" Jul 07 15:41:39.141367 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:39.141348 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9_e1db2e0e-d5ae-4949-83dd-8e2b07d50247/extract/0.log" Jul 07 15:41:40.176908 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:40.176874 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cjt6x_6e56a8b2-e0b9-4f3e-a5df-bb0688ae15fd/manager/0.log" Jul 07 15:41:40.556148 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:40.556067 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678mnt6z_0dbdce28-0c70-433d-a096-683323c424ca/istio-proxy/0.log" Jul 07 15:41:40.682782 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:40.682752 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5dwzk_d1f0acb2-e34e-4c9f-af49-c2557eec47c1/discovery/0.log" Jul 07 15:41:41.763947 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:41.763900 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-wv549_a6d462d9-8f38-47cc-bd7e-350fbf623d00/main/0.log" Jul 07 15:41:42.009009 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:42.008977 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-p9xdd_9031ede5-3faa-492e-912b-a2d4b15d9cba/main/0.log" Jul 07 15:41:49.491276 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:49.491233 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-66m5k_ae612123-dc47-43fd-b0aa-93d175975cd6/global-pull-secret-syncer/0.log" Jul 07 15:41:49.708133 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:49.708104 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-chsnd_28f013b7-2bc7-4ed9-9284-6ac5f5922566/konnectivity-agent/0.log" Jul 07 15:41:49.758569 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:49.758492 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-205.ec2.internal_0a252f3cefa1fdcdb223e8cd15c86c48/haproxy/0.log" Jul 07 15:41:54.599168 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:54.599126 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb_8a99e0df-82c6-4f71-b61c-41c8069311d2/extract/0.log" Jul 07 15:41:54.624972 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:54.624927 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb_8a99e0df-82c6-4f71-b61c-41c8069311d2/util/0.log" Jul 07 15:41:54.655356 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:54.655324 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592gjqb_8a99e0df-82c6-4f71-b61c-41c8069311d2/pull/0.log" Jul 07 15:41:54.682818 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:54.682790 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh_fb5faf61-8542-4065-a260-31669e2af05f/extract/0.log" Jul 07 15:41:54.709978 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:54.709952 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh_fb5faf61-8542-4065-a260-31669e2af05f/util/0.log" Jul 07 15:41:54.734684 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:54.734662 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06zmqh_fb5faf61-8542-4065-a260-31669e2af05f/pull/0.log" Jul 07 15:41:54.764159 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:54.764133 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw_f5cf3c2a-1a83-407e-bcad-a03da4e78d1c/extract/0.log" Jul 07 15:41:54.791012 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:54.790988 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw_f5cf3c2a-1a83-407e-bcad-a03da4e78d1c/util/0.log" Jul 07 15:41:54.816113 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:54.816085 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2zxw_f5cf3c2a-1a83-407e-bcad-a03da4e78d1c/pull/0.log" Jul 07 15:41:54.856459 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:54.856387 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9_e1db2e0e-d5ae-4949-83dd-8e2b07d50247/extract/0.log" Jul 07 15:41:54.884650 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:54.884626 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9_e1db2e0e-d5ae-4949-83dd-8e2b07d50247/util/0.log" Jul 07 15:41:54.905551 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:54.905528 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12g8k9_e1db2e0e-d5ae-4949-83dd-8e2b07d50247/pull/0.log" Jul 07 15:41:55.576522 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:55.576493 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cjt6x_6e56a8b2-e0b9-4f3e-a5df-bb0688ae15fd/manager/0.log" Jul 07 15:41:57.553485 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:57.553452 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-pcrj9_6c145f55-6798-421c-9d01-e6e42fe2a7f6/kube-state-metrics/0.log" Jul 07 15:41:57.588434 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:57.588407 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-pcrj9_6c145f55-6798-421c-9d01-e6e42fe2a7f6/kube-rbac-proxy-main/0.log" Jul 07 15:41:57.617623 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:57.617596 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-pcrj9_6c145f55-6798-421c-9d01-e6e42fe2a7f6/kube-rbac-proxy-self/0.log" Jul 07 15:41:57.648488 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:57.648461 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7bfd95bbff-dq4f5_8ef5f4ac-fcc0-4174-bf67-2b29456a4988/metrics-server/0.log" Jul 07 15:41:57.712979 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:57.712934 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ctxm9_c31fc1aa-2d1b-4e21-b2fc-603ce581ce50/node-exporter/0.log" Jul 07 15:41:57.736184 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:57.736160 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ctxm9_c31fc1aa-2d1b-4e21-b2fc-603ce581ce50/kube-rbac-proxy/0.log" Jul 07 15:41:57.762278 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:57.762254 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ctxm9_c31fc1aa-2d1b-4e21-b2fc-603ce581ce50/init-textfile/0.log" Jul 07 15:41:57.975664 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:57.975634 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-tdcz5_00f1ebed-3e42-4141-9c94-1b312a7c9c43/kube-rbac-proxy-main/0.log" Jul 07 15:41:58.002353 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:58.002329 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-tdcz5_00f1ebed-3e42-4141-9c94-1b312a7c9c43/kube-rbac-proxy-self/0.log" Jul 07 15:41:58.027351 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:58.027329 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-tdcz5_00f1ebed-3e42-4141-9c94-1b312a7c9c43/openshift-state-metrics/0.log" Jul 07 15:41:58.294638 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:58.294562 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-pnr2g_05477565-c157-422e-9691-b00ca6317b58/prometheus-operator-admission-webhook/0.log" Jul 07 15:41:58.338919 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:58.338889 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56485b6f5-ksg6v_29ba6248-6819-457a-b65a-6ba748d86a0a/telemeter-client/0.log" Jul 07 15:41:58.360589 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:58.360560 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56485b6f5-ksg6v_29ba6248-6819-457a-b65a-6ba748d86a0a/reload/0.log" Jul 07 15:41:58.386889 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:58.386855 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56485b6f5-ksg6v_29ba6248-6819-457a-b65a-6ba748d86a0a/kube-rbac-proxy/0.log" Jul 07 15:41:58.415575 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:58.415549 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67b5f99c8c-k666l_3e12f9e4-29a4-4381-b884-db2d849aed43/thanos-query/0.log" Jul 07 15:41:58.438135 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:58.438115 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67b5f99c8c-k666l_3e12f9e4-29a4-4381-b884-db2d849aed43/kube-rbac-proxy-web/0.log" Jul 07 15:41:58.459956 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:58.459917 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67b5f99c8c-k666l_3e12f9e4-29a4-4381-b884-db2d849aed43/kube-rbac-proxy/0.log" Jul 07 15:41:58.482093 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:58.482070 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67b5f99c8c-k666l_3e12f9e4-29a4-4381-b884-db2d849aed43/prom-label-proxy/0.log" Jul 07 15:41:58.507355 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:58.507333 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67b5f99c8c-k666l_3e12f9e4-29a4-4381-b884-db2d849aed43/kube-rbac-proxy-rules/0.log" Jul 07 15:41:58.529250 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:58.529231 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-67b5f99c8c-k666l_3e12f9e4-29a4-4381-b884-db2d849aed43/kube-rbac-proxy-metrics/0.log" Jul 07 15:41:59.890657 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.890622 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr"] Jul 07 15:41:59.891070 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.891021 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b33cdbcb-c13c-4fbe-a9a1-c409b965a486" containerName="maas-api" Jul 07 15:41:59.891070 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.891034 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b33cdbcb-c13c-4fbe-a9a1-c409b965a486" containerName="maas-api" Jul 07 15:41:59.891070 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.891045 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17d0d7a1-92c7-4f51-90c3-ccb19c0591c0" containerName="maas-api" Jul 07 15:41:59.891070 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.891050 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="17d0d7a1-92c7-4f51-90c3-ccb19c0591c0" containerName="maas-api" Jul 07 15:41:59.891070 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.891060 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ef1f01e-209c-43c2-a781-31ea117c8259" containerName="main" Jul 07 15:41:59.891070 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.891066 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ef1f01e-209c-43c2-a781-31ea117c8259" containerName="main" Jul 07 15:41:59.891269 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.891130 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b33cdbcb-c13c-4fbe-a9a1-c409b965a486" containerName="maas-api" Jul 07 15:41:59.891269 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.891137 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ef1f01e-209c-43c2-a781-31ea117c8259" containerName="main" Jul 07 15:41:59.891269 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.891142 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="17d0d7a1-92c7-4f51-90c3-ccb19c0591c0" containerName="maas-api" Jul 07 15:41:59.894430 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.894416 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:41:59.897007 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.896987 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hkchs\"/\"default-dockercfg-lzg8p\"" Jul 07 15:41:59.897007 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.896999 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hkchs\"/\"openshift-service-ca.crt\"" Jul 07 15:41:59.897842 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.897822 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hkchs\"/\"kube-root-ca.crt\"" Jul 07 15:41:59.902257 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.901920 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba82d4cf-3b05-4b03-8e20-f1fd10234658-podres\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:41:59.902257 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.901992 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba82d4cf-3b05-4b03-8e20-f1fd10234658-proc\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:41:59.902257 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.902044 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba82d4cf-3b05-4b03-8e20-f1fd10234658-lib-modules\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:41:59.902257 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.902086 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba82d4cf-3b05-4b03-8e20-f1fd10234658-sys\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:41:59.902257 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.902173 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6wg2\" (UniqueName: \"kubernetes.io/projected/ba82d4cf-3b05-4b03-8e20-f1fd10234658-kube-api-access-b6wg2\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:41:59.902669 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:41:59.902650 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr"] Jul 07 15:42:00.002745 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.002715 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6wg2\" (UniqueName: \"kubernetes.io/projected/ba82d4cf-3b05-4b03-8e20-f1fd10234658-kube-api-access-b6wg2\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:42:00.002918 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.002764 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba82d4cf-3b05-4b03-8e20-f1fd10234658-podres\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:42:00.002918 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.002790 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba82d4cf-3b05-4b03-8e20-f1fd10234658-proc\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:42:00.002918 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.002821 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba82d4cf-3b05-4b03-8e20-f1fd10234658-lib-modules\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:42:00.002918 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.002876 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba82d4cf-3b05-4b03-8e20-f1fd10234658-sys\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:42:00.003117 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.002979 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba82d4cf-3b05-4b03-8e20-f1fd10234658-lib-modules\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:42:00.003117 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.002983 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba82d4cf-3b05-4b03-8e20-f1fd10234658-podres\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:42:00.003117 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.002996 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba82d4cf-3b05-4b03-8e20-f1fd10234658-proc\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:42:00.003117 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.003014 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba82d4cf-3b05-4b03-8e20-f1fd10234658-sys\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:42:00.011635 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.011602 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6wg2\" (UniqueName: \"kubernetes.io/projected/ba82d4cf-3b05-4b03-8e20-f1fd10234658-kube-api-access-b6wg2\") pod \"perf-node-gather-daemonset-6q9dr\" (UID: \"ba82d4cf-3b05-4b03-8e20-f1fd10234658\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:42:00.204868 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.204830 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:42:00.340032 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.340007 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr"] Jul 07 15:42:00.341645 ip-10-0-141-205 kubenswrapper[2565]: W0707 15:42:00.341614 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podba82d4cf_3b05_4b03_8e20_f1fd10234658.slice/crio-3799c51a7e1d58cc25170319c6c342eef0034f4e4e3d50a172b21f77bc9aac76 WatchSource:0}: Error finding container 3799c51a7e1d58cc25170319c6c342eef0034f4e4e3d50a172b21f77bc9aac76: Status 404 returned error can't find the container with id 3799c51a7e1d58cc25170319c6c342eef0034f4e4e3d50a172b21f77bc9aac76 Jul 07 15:42:00.599962 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.599908 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" event={"ID":"ba82d4cf-3b05-4b03-8e20-f1fd10234658","Type":"ContainerStarted","Data":"04dfad24592f98466f143a448278ca2829280d6c85e425f2f9e5fa5cf3c5cb6b"} Jul 07 15:42:00.600161 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.599979 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:42:00.600161 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.599996 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" event={"ID":"ba82d4cf-3b05-4b03-8e20-f1fd10234658","Type":"ContainerStarted","Data":"3799c51a7e1d58cc25170319c6c342eef0034f4e4e3d50a172b21f77bc9aac76"} Jul 07 15:42:00.616370 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.616328 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" podStartSLOduration=1.616313015 podStartE2EDuration="1.616313015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:41:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:42:00.61552524 +0000 UTC m=+2450.040913628" watchObservedRunningTime="2026-07-07 15:42:00.616313015 +0000 UTC m=+2450.041701403" Jul 07 15:42:00.909154 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:00.909075 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-87747b946-zsdv6_50783695-e692-4b13-9108-0ff7e112a892/console/0.log" Jul 07 15:42:02.600776 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:02.600722 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6ghkd_e6139731-a6d0-45e4-b16a-a1743d67f6f1/dns/0.log" Jul 07 15:42:02.627033 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:02.627008 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6ghkd_e6139731-a6d0-45e4-b16a-a1743d67f6f1/kube-rbac-proxy/0.log" Jul 07 15:42:02.761108 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:02.761078 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6wgcw_bf1e3894-e18e-4ea6-bb7b-b7ab0338c85d/dns-node-resolver/0.log" Jul 07 15:42:03.437510 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:03.437468 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ssg6q_395c28b2-8a48-403f-81b0-51e8e602646d/node-ca/0.log" Jul 07 15:42:04.429667 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:04.429638 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678mnt6z_0dbdce28-0c70-433d-a096-683323c424ca/istio-proxy/0.log" Jul 07 15:42:04.542685 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:04.542655 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5dwzk_d1f0acb2-e34e-4c9f-af49-c2557eec47c1/discovery/0.log" Jul 07 15:42:05.523546 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:05.523520 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dvpnf_d0ca0927-1d4e-4dc0-a065-68a8d745cdc3/serve-healthcheck-canary/0.log" Jul 07 15:42:06.247767 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:06.247740 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zhxj5_141b7ee7-5826-40f9-80ba-65febbba86ca/kube-rbac-proxy/0.log" Jul 07 15:42:06.273210 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:06.273188 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zhxj5_141b7ee7-5826-40f9-80ba-65febbba86ca/exporter/0.log" Jul 07 15:42:06.300599 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:06.300575 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zhxj5_141b7ee7-5826-40f9-80ba-65febbba86ca/extractor/0.log" Jul 07 15:42:06.614443 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:06.614372 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-6q9dr" Jul 07 15:42:08.986576 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:08.986542 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723970-d5bxv_9526c65e-7436-41dc-8a8b-afe7a07b0dad/cleanup/0.log" Jul 07 15:42:09.258007 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:09.257974 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845dc9964f-2w2h8_10a2db9b-1d35-4dfc-af70-cbf80dd72ff6/manager/0.log" Jul 07 15:42:09.284016 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:09.283986 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845dc9964f-2w2h8_10a2db9b-1d35-4dfc-af70-cbf80dd72ff6/copy-manifests/0.log" Jul 07 15:42:10.441385 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:10.441352 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678mnt6z_0dbdce28-0c70-433d-a096-683323c424ca/istio-proxy/0.log" Jul 07 15:42:10.539794 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:10.539757 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5dwzk_d1f0acb2-e34e-4c9f-af49-c2557eec47c1/discovery/0.log" Jul 07 15:42:10.892762 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:10.892665 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d59ff9cf9-4mcth_f67887fd-d0ea-47e4-a61e-f4208b204211/manager/0.log" Jul 07 15:42:12.033089 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:12.032986 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678mnt6z_0dbdce28-0c70-433d-a096-683323c424ca/istio-proxy/0.log" Jul 07 15:42:12.139226 ip-10-0-141-205 kubenswrapper[2565]: I0707 15:42:12.139191 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5dwzk_d1f0acb2-e34e-4c9f-af49-c2557eec47c1/discovery/0.log"