Jul 06 21:11:07.090021 ip-10-0-136-12 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 21:11:07.090039 ip-10-0-136-12 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 21:11:07.090049 ip-10-0-136-12 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 21:11:07.090488 ip-10-0-136-12 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 21:11:17.281289 ip-10-0-136-12 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 21:11:17.281309 ip-10-0-136-12 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 74847e54c0994b4b899ceb7ce538b124 -- Jul 06 21:12:55.751811 ip-10-0-136-12 systemd[1]: Starting Kubernetes Kubelet... Jul 06 21:12:56.291768 ip-10-0-136-12 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 21:12:56.291768 ip-10-0-136-12 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 21:12:56.291768 ip-10-0-136-12 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 21:12:56.291768 ip-10-0-136-12 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 21:12:56.291768 ip-10-0-136-12 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 21:12:56.291768 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.181965 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 21:12:56.291768 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.188991 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 21:12:56.291768 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189006 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 21:12:56.291768 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189011 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 21:12:56.291768 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189016 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 21:12:56.291768 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189020 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 21:12:56.291768 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189027 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 21:12:56.291768 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189031 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189037 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189043 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189048 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189052 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189057 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189061 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189065 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189069 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189073 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189077 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189081 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189084 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189087 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189098 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189102 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189105 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189109 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189113 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 21:12:56.296874 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189117 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 21:12:56.262921 ip-10-0-136-12 systemd[1]: Started Kubernetes Kubelet. Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189121 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189126 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189130 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189133 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189137 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189141 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189145 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189149 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189153 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189157 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189162 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189166 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189170 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189173 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189177 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189189 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189193 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189197 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189201 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 21:12:56.300960 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189205 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189208 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189213 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189217 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189221 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189224 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189228 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189232 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189236 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189240 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189243 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189247 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189251 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189255 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189259 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189264 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189268 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189272 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189276 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189280 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 21:12:56.305490 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189283 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189287 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189291 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189295 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189298 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189302 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189306 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189309 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189315 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189321 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189324 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189328 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189331 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189335 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189341 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189347 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189350 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189354 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189359 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 21:12:56.307440 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189362 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189366 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189370 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189374 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189378 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189381 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189385 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189391 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189396 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189400 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189404 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189408 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189412 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189417 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.189421 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190427 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190439 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190444 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190450 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190453 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 21:12:56.308252 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190458 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190480 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190485 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190489 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190494 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190498 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190502 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190506 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190510 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190515 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190520 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190524 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190527 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190533 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190537 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190542 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190546 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190549 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190553 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190557 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 21:12:56.311923 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190561 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190567 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190572 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190576 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190579 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190583 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190587 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190591 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190595 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190598 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190602 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190606 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190610 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190613 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190618 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190622 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190626 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190630 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190633 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190637 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 21:12:56.314480 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190641 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190644 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190649 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190653 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190658 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190663 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190667 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190672 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190676 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190680 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190683 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190687 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190693 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190696 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190700 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190706 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190710 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190714 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190718 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 21:12:56.316881 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190722 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190726 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190729 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190732 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190736 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190740 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190744 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190747 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190751 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190755 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190759 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190762 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190765 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190769 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190773 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190777 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190781 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190784 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190788 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190792 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 21:12:56.318949 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190795 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190799 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190802 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190808 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190811 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190815 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190819 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190823 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190828 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190831 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190835 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190839 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190843 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190847 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.190851 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191383 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191388 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191392 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191396 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191400 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 21:12:56.321457 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191404 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191408 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191412 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191416 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191419 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191423 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191428 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191431 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191436 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191440 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191444 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191448 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191451 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191455 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191459 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191483 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191487 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191492 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191497 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 21:12:56.322674 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191501 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191507 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191511 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191515 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191523 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191528 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191532 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191537 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191541 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191545 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191548 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191552 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191558 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191562 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191566 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191570 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191574 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191578 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191581 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191585 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 21:12:56.324223 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191588 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191592 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191596 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191600 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191604 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191608 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191612 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191616 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191622 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191626 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191629 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191632 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191636 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191643 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191647 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191651 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191654 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191658 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191661 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191665 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 21:12:56.325021 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191668 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191674 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191678 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191682 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191686 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191690 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191694 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191698 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191701 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191705 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191709 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191712 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191716 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191719 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191723 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191728 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191731 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191735 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191739 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191742 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 21:12:56.327433 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191748 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191752 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191755 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191759 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191764 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191768 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191772 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191775 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191779 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191783 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191787 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191791 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191794 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191798 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.191801 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191895 2577 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191904 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191913 2577 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191919 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191924 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191929 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 21:12:56.328278 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191934 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191940 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191945 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191950 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191954 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191959 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191963 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191968 2577 flags.go:64] FLAG: --cgroup-root="" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191972 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191977 2577 flags.go:64] FLAG: --client-ca-file="" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191981 2577 flags.go:64] FLAG: --cloud-provider="external" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191987 2577 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191993 2577 flags.go:64] FLAG: --cluster-domain="" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.191998 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192002 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192006 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192012 2577 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192017 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192022 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192028 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192032 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192036 2577 flags.go:64] FLAG: --contention-profiling="false" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192041 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192045 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 21:12:56.329892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192049 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192054 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192060 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192065 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192069 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192073 2577 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192077 2577 flags.go:64] FLAG: --enable-server="true" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192082 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192087 2577 flags.go:64] FLAG: --event-burst="100" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192092 2577 flags.go:64] FLAG: --event-qps="50" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192096 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192101 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192105 2577 flags.go:64] FLAG: --eviction-hard="" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192111 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192115 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192120 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192125 2577 flags.go:64] FLAG: --eviction-soft="" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192129 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192133 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192139 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192144 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192148 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192153 2577 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192157 2577 flags.go:64] FLAG: --feature-gates="" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192165 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 21:12:56.330690 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192169 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192173 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192178 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192182 2577 flags.go:64] FLAG: --healthz-port="10248" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192186 2577 flags.go:64] FLAG: --help="false" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192191 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192196 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192200 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192204 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192209 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192215 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192218 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192222 2577 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192227 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192231 2577 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192235 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192240 2577 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192244 2577 flags.go:64] FLAG: --kube-reserved="" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192249 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192253 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192257 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192262 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192266 2577 flags.go:64] FLAG: --lock-file="" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192269 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 21:12:56.331898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192274 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192279 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192288 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192292 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192296 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192301 2577 flags.go:64] FLAG: --logging-format="text" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192305 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192311 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192315 2577 flags.go:64] FLAG: --manifest-url="" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192319 2577 flags.go:64] FLAG: --manifest-url-header="" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192325 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192330 2577 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192336 2577 flags.go:64] FLAG: --max-pods="110" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192340 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192345 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192349 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192354 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192358 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192362 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192367 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192377 2577 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192381 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192385 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 21:12:56.332697 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192390 2577 flags.go:64] FLAG: --pod-cidr="" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192394 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192402 2577 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192406 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192410 2577 flags.go:64] FLAG: --pods-per-core="0" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192415 2577 flags.go:64] FLAG: --port="10250" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192419 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192423 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-019630ab49d64f948" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192427 2577 flags.go:64] FLAG: --qos-reserved="" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192433 2577 flags.go:64] FLAG: --read-only-port="10255" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192438 2577 flags.go:64] FLAG: --register-node="true" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192442 2577 flags.go:64] FLAG: --register-with-taints="" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192448 2577 flags.go:64] FLAG: --registry-burst="10" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192452 2577 flags.go:64] FLAG: --registry-qps="5" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192456 2577 flags.go:64] FLAG: --reserved-cpus="" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192460 2577 flags.go:64] FLAG: --reserved-memory="" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192482 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192486 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192491 2577 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192496 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192500 2577 flags.go:64] FLAG: --runonce="false" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192504 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192509 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192513 2577 flags.go:64] FLAG: --seccomp-default="false" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192517 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 21:12:56.335945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192521 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192527 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192531 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192536 2577 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192540 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192544 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192548 2577 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192552 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192557 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192561 2577 flags.go:64] FLAG: --system-cgroups="" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192598 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192609 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192613 2577 flags.go:64] FLAG: --tls-cert-file="" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192616 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192623 2577 flags.go:64] FLAG: --tls-min-version="" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192626 2577 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192630 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192633 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192636 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192639 2577 flags.go:64] FLAG: --v="2" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192652 2577 flags.go:64] FLAG: --version="false" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192656 2577 flags.go:64] FLAG: --vmodule="" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192660 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.192664 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192802 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 21:12:56.337132 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192808 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192811 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192814 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192816 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192819 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192823 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192826 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192830 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192834 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192837 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192840 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192843 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192846 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192849 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192852 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192859 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192862 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192865 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192868 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 21:12:56.338742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192871 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192874 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192877 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192881 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192885 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192888 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192890 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192893 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192896 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192899 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192902 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192905 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192908 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192911 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192914 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192916 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192919 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192921 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192924 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192927 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 21:12:56.339557 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192929 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192932 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192935 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192937 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192940 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192942 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192945 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192947 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192951 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192954 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192957 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192959 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192963 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192966 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192968 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192971 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192973 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192976 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192978 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192981 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 21:12:56.352111 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192983 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192987 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192989 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192992 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192994 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.192997 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193000 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193002 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193005 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193008 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193012 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193014 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193017 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193020 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193023 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193025 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193028 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193030 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193033 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193036 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 21:12:56.352903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193040 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193043 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193045 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193048 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193051 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193054 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193056 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193059 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193062 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193064 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193067 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193069 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193072 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193074 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193077 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193081 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193084 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193087 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 21:12:56.507442 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.193090 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.193095 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.201382 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.201396 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.202237 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.202244 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202301 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202306 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202309 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202313 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202316 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202319 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202321 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202325 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 21:12:56.526870 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202327 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202332 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202335 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202338 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202340 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202343 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202346 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202349 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202351 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202354 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202356 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202359 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202362 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202365 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202367 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202370 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202373 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202375 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202378 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202380 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 21:12:56.529280 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202383 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202386 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202390 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202392 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202395 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202397 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202400 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202403 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202405 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202408 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202412 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202416 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202419 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202423 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202426 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202429 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202432 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202434 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202437 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202439 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 21:12:56.530056 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202442 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202445 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202447 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202450 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202453 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202456 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202458 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202475 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202479 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202482 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202484 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202487 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202490 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202494 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202499 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202502 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202505 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202507 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202510 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202513 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 21:12:56.532482 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202515 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202518 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202520 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202523 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202525 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202529 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202532 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202535 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202537 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202540 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202542 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202546 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202548 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202551 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202554 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202556 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202559 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202561 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202564 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202566 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 21:12:56.533258 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202569 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202571 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202574 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202577 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202579 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202582 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202586 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202588 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202592 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202596 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202598 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.202604 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202730 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202736 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 21:12:56.536391 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202739 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202742 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202745 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202747 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202751 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202755 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202758 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202761 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202763 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202766 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202770 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202772 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202775 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202778 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202781 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202783 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202786 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202788 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202791 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202793 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 21:12:56.537173 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202796 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202799 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202801 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202804 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202808 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202810 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202813 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202816 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202818 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202821 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202823 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202826 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202828 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202831 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202833 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202836 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202840 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202843 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202846 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202849 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 21:12:56.539939 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202851 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202854 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202857 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202860 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202863 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202866 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202868 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202870 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202873 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202875 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202878 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202880 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202883 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202885 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202888 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202891 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202895 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202897 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202900 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 21:12:56.540742 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202903 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202906 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202910 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202914 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202916 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202919 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202922 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202925 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202927 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202932 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202935 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202938 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202942 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202945 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202947 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202950 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202953 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202956 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202959 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 21:12:56.544348 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202962 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202965 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202967 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202970 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202972 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202975 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202977 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202980 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202983 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202985 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202989 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202992 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202995 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.202997 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.203000 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.203002 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.203005 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.203008 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 21:12:56.545144 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:56.203010 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 21:12:56.549915 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.203015 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 21:12:56.549915 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.203125 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 21:12:56.549915 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.205780 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 21:12:56.549915 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.206956 2577 server.go:1019] "Starting client certificate rotation" Jul 06 21:12:56.549915 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.207070 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 21:12:56.549915 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.207117 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 21:12:56.549915 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.210287 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 21:12:56.549915 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.211406 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 21:12:56.549915 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.225422 2577 log.go:25] "Validated CRI v1 runtime API" Jul 06 21:12:56.549915 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.228665 2577 log.go:25] "Validated CRI v1 image API" Jul 06 21:12:56.549915 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.230146 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 21:12:56.549915 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.233554 2577 fs.go:135] Filesystem UUIDs: map[25b49af7-5af7-43f8-986d-4151af4136ed:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d261393b-2746-4f9d-ae0e-f3bc25036ea2:/dev/nvme0n1p3] Jul 06 21:12:56.550311 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.233572 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 21:12:56.550311 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.236317 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.240869 2577 manager.go:216] Machine: {Timestamp:2026-07-06 21:12:56.239496503 +0000 UTC m=+0.364477918 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3094467 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21028b8b20a0474ff2e8fc79708460 SystemUUID:ec21028b-8b20-a047-4ff2-e8fc79708460 BootID:74847e54-c099-4b4b-899c-eb7ce538b124 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:64:3b:5e:89:1f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:64:3b:5e:89:1f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:b4:96:7b:7b:6f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.240969 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.241055 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.244219 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.244240 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-12.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.245620 2577 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.245632 2577 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.245640 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.245656 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.247156 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.250411 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.250540 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.250569 2577 kubelet.go:486] "Attempting to sync node with API server" Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.250583 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.250619 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.250627 2577 kubelet.go:398] "Adding apiserver pod source" Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.250636 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 21:12:56.553508 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.251726 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.251763 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.254270 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.256076 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c2n28" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.256240 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.256276 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.257388 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.257413 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.257421 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.257428 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.257443 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.257450 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.257473 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.257482 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.257502 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.257513 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.257524 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.257531 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.258242 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.258251 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.262008 2577 server.go:1268] "Started kubelet" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.262097 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.262560 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-12.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.262684 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-12.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.262702 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.262730 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.262789 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.263761 2577 server.go:310] "Adding debug handlers to kubelet server" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.263973 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.265889 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c2n28" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.267892 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.267260 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-12.ec2.internal.18bfcfba3d56ee07 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-12.ec2.internal,UID:ip-10-0-136-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-12.ec2.internal,},FirstTimestamp:2026-07-06 21:12:56.261979655 +0000 UTC m=+0.386961070,LastTimestamp:2026-07-06 21:12:56.261979655 +0000 UTC m=+0.386961070,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-12.ec2.internal,}" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.270541 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.270546 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.270672 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.271135 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.271155 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.271134 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.271285 2577 reconstruct.go:97] "Volume reconstruction finished" Jul 06 21:12:56.556270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.271293 2577 reconciler.go:29] "Reconciler: start to sync state" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.272099 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.272112 2577 factory.go:55] Registering systemd factory Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.272121 2577 factory.go:223] Registration of the systemd container factory successfully Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.273232 2577 factory.go:153] Registering CRI-O factory Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.273245 2577 factory.go:223] Registration of the crio container factory successfully Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.273268 2577 factory.go:103] Registering Raw factory Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.273280 2577 manager.go:1196] Started watching for new ooms in manager Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.273796 2577 manager.go:349] Starting recovery of all containers Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.276888 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.282620 2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-12.ec2.internal\" not found" node="ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.286719 2577 manager.go:354] Recovery completed Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.290925 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.292981 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.293010 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.293025 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.293559 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.293570 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.293573 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.293589 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.294239 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.294268 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.294282 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.294810 2577 policy_none.go:49] "None policy: Start" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.294822 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.294832 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.295857 2577 policy_none.go:47] "Start" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.370763 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.389089 2577 manager.go:337] "Starting Device Plugin manager" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.389119 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.389128 2577 server.go:87] "Starting device plugin registration server" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.389394 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.389404 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.389759 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.389852 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.389861 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.390400 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.390442 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.482368 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.483486 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.483520 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.483545 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.483582 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.486082 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.490008 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.491209 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.491234 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.491244 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.491270 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.557490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.499255 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.562285 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.499274 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-12.ec2.internal\": node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:56.562285 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.514123 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.583928 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-12.ec2.internal"] Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.584015 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.585593 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.585628 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.585644 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.586907 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.587014 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.587052 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.587703 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.587707 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.587758 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.587774 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.587730 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.587797 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.588913 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.588943 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.589648 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.589675 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.589689 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:12:56.663888 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.614320 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:56.674626 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.673109 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d463b2868e96d13a950b18acfec1156-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal\" (UID: \"4d463b2868e96d13a950b18acfec1156\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.674626 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.673142 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d463b2868e96d13a950b18acfec1156-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal\" (UID: \"4d463b2868e96d13a950b18acfec1156\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.674626 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.673166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/54f38d8f384b65c8b8874f57c20a13d4-config\") pod \"kube-apiserver-proxy-ip-10-0-136-12.ec2.internal\" (UID: \"54f38d8f384b65c8b8874f57c20a13d4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.784416 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.715284 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:56.784416 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.773730 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/54f38d8f384b65c8b8874f57c20a13d4-config\") pod \"kube-apiserver-proxy-ip-10-0-136-12.ec2.internal\" (UID: \"54f38d8f384b65c8b8874f57c20a13d4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.784416 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.773765 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d463b2868e96d13a950b18acfec1156-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal\" (UID: \"4d463b2868e96d13a950b18acfec1156\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.784416 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.773779 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d463b2868e96d13a950b18acfec1156-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal\" (UID: \"4d463b2868e96d13a950b18acfec1156\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.784416 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.773813 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/54f38d8f384b65c8b8874f57c20a13d4-config\") pod \"kube-apiserver-proxy-ip-10-0-136-12.ec2.internal\" (UID: \"54f38d8f384b65c8b8874f57c20a13d4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.784416 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.773841 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d463b2868e96d13a950b18acfec1156-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal\" (UID: \"4d463b2868e96d13a950b18acfec1156\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.784416 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:56.773821 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d463b2868e96d13a950b18acfec1156-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal\" (UID: \"4d463b2868e96d13a950b18acfec1156\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal" Jul 06 21:12:56.913562 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.815892 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:56.917245 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:56.916653 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:57.017327 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:57.017291 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:57.033667 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:57.033647 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-12.ec2.internal\" not found" node="ip-10-0-136-12.ec2.internal" Jul 06 21:12:57.036839 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.036822 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal" Jul 06 21:12:57.038662 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:57.038645 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-12.ec2.internal\" not found" node="ip-10-0-136-12.ec2.internal" Jul 06 21:12:57.041414 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.041398 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-12.ec2.internal" Jul 06 21:12:57.119183 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:57.119148 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:57.208378 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.208285 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 21:12:57.208551 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.208465 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 21:12:57.208551 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.208524 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 21:12:57.220322 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:57.220296 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:57.271784 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.271742 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 21:07:56 +0000 UTC" deadline="2028-04-11 14:09:36.917173854 +0000 UTC" Jul 06 21:12:57.271784 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.271775 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15472h56m39.645403699s" Jul 06 21:12:57.323227 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:57.323197 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:57.423586 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:57.423555 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-12.ec2.internal\" not found" Jul 06 21:12:57.434490 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.434466 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 21:12:57.455035 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.455010 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 21:12:57.471823 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.471751 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal" Jul 06 21:12:57.486656 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.486629 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 21:12:57.487874 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.487859 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-12.ec2.internal" Jul 06 21:12:57.501868 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.501807 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 21:12:57.616351 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:57.616322 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d463b2868e96d13a950b18acfec1156.slice/crio-8413dfa4ce3db50cd40fd9087b5b75c9b46c1cce694e98f18d7df6c2da98aa7d WatchSource:0}: Error finding container 8413dfa4ce3db50cd40fd9087b5b75c9b46c1cce694e98f18d7df6c2da98aa7d: Status 404 returned error can't find the container with id 8413dfa4ce3db50cd40fd9087b5b75c9b46c1cce694e98f18d7df6c2da98aa7d Jul 06 21:12:57.616680 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:57.616658 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f38d8f384b65c8b8874f57c20a13d4.slice/crio-15f1ee738918e2727f1a7d9e83cf594c71233cf750ee408846b6a9c3e8c5317a WatchSource:0}: Error finding container 15f1ee738918e2727f1a7d9e83cf594c71233cf750ee408846b6a9c3e8c5317a: Status 404 returned error can't find the container with id 15f1ee738918e2727f1a7d9e83cf594c71233cf750ee408846b6a9c3e8c5317a Jul 06 21:12:57.621384 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:57.621366 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 21:12:58.014917 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.014885 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 21:12:58.253442 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.253413 2577 apiserver.go:52] "Watching apiserver" Jul 06 21:12:58.265066 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.264983 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 21:12:58.265503 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.265470 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-mzmmk","openshift-network-operator/iptables-alerter-98smj","kube-system/konnectivity-agent-ddkph","kube-system/kube-apiserver-proxy-ip-10-0-136-12.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal","openshift-multus/multus-additional-cni-plugins-7ds6m","openshift-multus/multus-h4886","openshift-multus/network-metrics-daemon-xv8m9","openshift-ovn-kubernetes/ovnkube-node-g64lt","kube-system/global-pull-secret-syncer-xjshg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w","openshift-cluster-node-tuning-operator/tuned-crh49","openshift-dns/node-resolver-r9fcz","openshift-image-registry/node-ca-8w488"] Jul 06 21:12:58.267436 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.267179 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:12:58.267436 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:58.267263 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:12:58.268932 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.268666 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.269954 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.269867 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ddkph" Jul 06 21:12:58.271237 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.271214 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 21:12:58.271589 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.271569 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 21:12:58.272140 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.271790 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 21:12:58.272231 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.272156 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 21:12:58.272382 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.272363 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 21:12:58.272994 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.272975 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9hx8t\"" Jul 06 21:12:58.273073 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.273029 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 21:12:58.273073 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.273061 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zftbz\"" Jul 06 21:12:58.273287 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.273245 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8w488" Jul 06 21:12:58.274756 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.274503 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:12:58.274756 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:58.274586 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:12:58.276186 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.275875 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.281165 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.280921 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 21:12:58.281165 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.281004 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 21:12:58.281165 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.281126 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 21:12:58.281342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.281201 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 21:12:58.281397 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.281382 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 21:12:58.281451 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.281409 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 21:12:58.281706 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.281612 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 21:12:58.281706 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.281665 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-thztn\"" Jul 06 21:12:58.281910 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.281837 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-d4qxs\"" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.283857 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.284852 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-device-dir\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.284895 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rszm9\" (UniqueName: \"kubernetes.io/projected/b490d547-2ba8-44f0-a6e6-898846f59cc9-kube-api-access-rszm9\") pod \"node-ca-8w488\" (UID: \"b490d547-2ba8-44f0-a6e6-898846f59cc9\") " pod="openshift-image-registry/node-ca-8w488" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.284938 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b490d547-2ba8-44f0-a6e6-898846f59cc9-serviceca\") pod \"node-ca-8w488\" (UID: \"b490d547-2ba8-44f0-a6e6-898846f59cc9\") " pod="openshift-image-registry/node-ca-8w488" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.284973 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c8194ca3-5e42-4623-aabb-091c40445eaa-dbus\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.285027 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.285068 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-sys-fs\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.285098 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2fc778ca-01e3-41a8-87ae-9e5d28afcaad-agent-certs\") pod \"konnectivity-agent-ddkph\" (UID: \"2fc778ca-01e3-41a8-87ae-9e5d28afcaad\") " pod="kube-system/konnectivity-agent-ddkph" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.285131 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b490d547-2ba8-44f0-a6e6-898846f59cc9-host\") pod \"node-ca-8w488\" (UID: \"b490d547-2ba8-44f0-a6e6-898846f59cc9\") " pod="openshift-image-registry/node-ca-8w488" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.285193 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.285219 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-etc-selinux\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.285249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtt2x\" (UniqueName: \"kubernetes.io/projected/b64ab87a-5226-488c-a68a-0f2d12f20b07-kube-api-access-qtt2x\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.285278 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2fc778ca-01e3-41a8-87ae-9e5d28afcaad-konnectivity-ca\") pod \"konnectivity-agent-ddkph\" (UID: \"2fc778ca-01e3-41a8-87ae-9e5d28afcaad\") " pod="kube-system/konnectivity-agent-ddkph" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.285309 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c8194ca3-5e42-4623-aabb-091c40445eaa-kubelet-config\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.285347 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-socket-dir\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.286342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.285389 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-registration-dir\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.287274 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.287166 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.291801 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.291248 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7rm8s\"" Jul 06 21:12:58.291801 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.291304 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 21:12:58.291801 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.291483 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r9fcz" Jul 06 21:12:58.291801 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.291491 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 21:12:58.294594 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.292727 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 21:12:58.294594 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.292842 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 21:12:58.294594 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.293007 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-98smj" Jul 06 21:12:58.294594 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.293058 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 21:12:58.294594 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.293161 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 21:12:58.294594 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.294167 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 21:12:58.294594 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.294186 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:12:58.294594 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:58.294254 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:12:58.297572 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.295274 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 21:12:58.297572 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.295355 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 21:12:58.297572 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.295621 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wrczr\"" Jul 06 21:12:58.297572 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.295671 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 21:12:58.297572 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.295813 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 21:12:58.297572 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.296180 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 21:12:58.297572 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.296196 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j2htp\"" Jul 06 21:12:58.297572 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.296246 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h4886" Jul 06 21:12:58.297572 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.297465 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.299224 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.299186 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 21:12:58.299484 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.299464 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ln4tj\"" Jul 06 21:12:58.300076 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.300054 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 21:12:58.300403 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.300384 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 21:12:58.300517 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.300431 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qq67k\"" Jul 06 21:12:58.321095 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.321067 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-brf95" Jul 06 21:12:58.334885 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.334859 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-brf95" Jul 06 21:12:58.373480 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.373450 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 21:12:58.386403 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386371 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-sysctl-d\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.386575 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386406 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-run\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.386575 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386437 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-sys\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.386575 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386457 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-lib-modules\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.386575 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386496 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs\") pod \"network-metrics-daemon-xv8m9\" (UID: \"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150\") " pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:12:58.386575 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386521 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-var-lib-kubelet\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.386575 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386546 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.386575 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386570 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwb8w\" (UniqueName: \"kubernetes.io/projected/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-kube-api-access-nwb8w\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.386925 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386598 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-kubelet\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.386925 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386623 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-etc-openvswitch\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.386925 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386650 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.386925 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386696 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6690861-f325-41b3-bf8c-5e43798ec3d0-ovn-node-metrics-cert\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.386925 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386759 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-socket-dir\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.386925 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386791 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rszm9\" (UniqueName: \"kubernetes.io/projected/b490d547-2ba8-44f0-a6e6-898846f59cc9-kube-api-access-rszm9\") pod \"node-ca-8w488\" (UID: \"b490d547-2ba8-44f0-a6e6-898846f59cc9\") " pod="openshift-image-registry/node-ca-8w488" Jul 06 21:12:58.386925 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386813 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6kj4\" (UniqueName: \"kubernetes.io/projected/b39d0a10-298e-4d2b-8469-0194658c0466-kube-api-access-k6kj4\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.386925 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386847 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/701babd3-9d0a-462d-b559-7508529a25f4-etc-tuned\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.386925 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386901 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-systemd-units\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.386925 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386926 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knrbw\" (UniqueName: \"kubernetes.io/projected/701babd3-9d0a-462d-b559-7508529a25f4-kube-api-access-knrbw\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.387399 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386937 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-socket-dir\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.387399 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386966 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-hostroot\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.387399 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.386993 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-run-systemd\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.387399 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387027 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c6690861-f325-41b3-bf8c-5e43798ec3d0-ovnkube-script-lib\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.387399 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387057 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2fc778ca-01e3-41a8-87ae-9e5d28afcaad-agent-certs\") pod \"konnectivity-agent-ddkph\" (UID: \"2fc778ca-01e3-41a8-87ae-9e5d28afcaad\") " pod="kube-system/konnectivity-agent-ddkph" Jul 06 21:12:58.387399 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387082 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-cnibin\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.387399 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387106 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c6690861-f325-41b3-bf8c-5e43798ec3d0-ovnkube-config\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.387399 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387133 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2fc778ca-01e3-41a8-87ae-9e5d28afcaad-konnectivity-ca\") pod \"konnectivity-agent-ddkph\" (UID: \"2fc778ca-01e3-41a8-87ae-9e5d28afcaad\") " pod="kube-system/konnectivity-agent-ddkph" Jul 06 21:12:58.387399 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387159 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-slash\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.387819 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387612 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 21:12:58.387819 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387796 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2fc778ca-01e3-41a8-87ae-9e5d28afcaad-konnectivity-ca\") pod \"konnectivity-agent-ddkph\" (UID: \"2fc778ca-01e3-41a8-87ae-9e5d28afcaad\") " pod="kube-system/konnectivity-agent-ddkph" Jul 06 21:12:58.387912 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387858 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-cni-netd\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.387912 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387892 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c8194ca3-5e42-4623-aabb-091c40445eaa-kubelet-config\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:12:58.388012 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387917 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-var-lib-kubelet\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.388012 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387943 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-host\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.388012 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387967 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/701babd3-9d0a-462d-b559-7508529a25f4-tmp\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.388012 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.387994 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a1be0cb-488e-4e55-855e-06abda43369b-hosts-file\") pod \"node-resolver-r9fcz\" (UID: \"6a1be0cb-488e-4e55-855e-06abda43369b\") " pod="openshift-dns/node-resolver-r9fcz" Jul 06 21:12:58.388203 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388019 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6753e71a-7591-4dca-b1ba-236c3e51c791-host-slash\") pod \"iptables-alerter-98smj\" (UID: \"6753e71a-7591-4dca-b1ba-236c3e51c791\") " pod="openshift-network-operator/iptables-alerter-98smj" Jul 06 21:12:58.388203 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388085 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.388203 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388103 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c8194ca3-5e42-4623-aabb-091c40445eaa-kubelet-config\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:12:58.388203 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388159 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c6690861-f325-41b3-bf8c-5e43798ec3d0-env-overrides\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.388203 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388191 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c8194ca3-5e42-4623-aabb-091c40445eaa-dbus\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388236 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388287 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388331 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-sys-fs\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388378 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b490d547-2ba8-44f0-a6e6-898846f59cc9-host\") pod \"node-ca-8w488\" (UID: \"b490d547-2ba8-44f0-a6e6-898846f59cc9\") " pod="openshift-image-registry/node-ca-8w488" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388407 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-sysctl-conf\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388448 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a1be0cb-488e-4e55-855e-06abda43369b-tmp-dir\") pod \"node-resolver-r9fcz\" (UID: \"6a1be0cb-488e-4e55-855e-06abda43369b\") " pod="openshift-dns/node-resolver-r9fcz" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388474 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-var-lib-cni-bin\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388518 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-run-multus-certs\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388569 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-run-netns\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388596 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-log-socket\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388619 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-registration-dir\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388638 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-device-dir\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388665 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-multus-cni-dir\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388690 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-etc-kubernetes\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.388840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388787 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-os-release\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388865 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-run-openvswitch\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388871 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c8194ca3-5e42-4623-aabb-091c40445eaa-dbus\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388896 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-run-ovn\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388925 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b490d547-2ba8-44f0-a6e6-898846f59cc9-serviceca\") pod \"node-ca-8w488\" (UID: \"b490d547-2ba8-44f0-a6e6-898846f59cc9\") " pod="openshift-image-registry/node-ca-8w488" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388952 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-sys-fs\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388970 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-run-netns\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388992 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b490d547-2ba8-44f0-a6e6-898846f59cc9-host\") pod \"node-ca-8w488\" (UID: \"b490d547-2ba8-44f0-a6e6-898846f59cc9\") " pod="openshift-image-registry/node-ca-8w488" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.388999 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-var-lib-cni-multus\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389032 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-sysconfig\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389084 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-systemd\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389105 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-registration-dir\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389125 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-cni-bin\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389153 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-device-dir\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzzjb\" (UniqueName: \"kubernetes.io/projected/c6690861-f325-41b3-bf8c-5e43798ec3d0-kube-api-access-gzzjb\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389215 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6753e71a-7591-4dca-b1ba-236c3e51c791-iptables-alerter-script\") pod \"iptables-alerter-98smj\" (UID: \"6753e71a-7591-4dca-b1ba-236c3e51c791\") " pod="openshift-network-operator/iptables-alerter-98smj" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-system-cni-dir\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.389838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389282 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-system-cni-dir\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389340 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-cni-binary-copy\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389368 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389398 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-node-log\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389429 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389499 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-etc-selinux\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389529 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtt2x\" (UniqueName: \"kubernetes.io/projected/b64ab87a-5226-488c-a68a-0f2d12f20b07-kube-api-access-qtt2x\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389573 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-os-release\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389602 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-multus-conf-dir\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389620 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b490d547-2ba8-44f0-a6e6-898846f59cc9-serviceca\") pod \"node-ca-8w488\" (UID: \"b490d547-2ba8-44f0-a6e6-898846f59cc9\") " pod="openshift-image-registry/node-ca-8w488" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389629 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-modprobe-d\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389666 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-kubernetes\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389697 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-run-ovn-kubernetes\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389725 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqggq\" (UniqueName: \"kubernetes.io/projected/6753e71a-7591-4dca-b1ba-236c3e51c791-kube-api-access-xqggq\") pod \"iptables-alerter-98smj\" (UID: \"6753e71a-7591-4dca-b1ba-236c3e51c791\") " pod="openshift-network-operator/iptables-alerter-98smj" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389753 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-multus-socket-dir-parent\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389754 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b64ab87a-5226-488c-a68a-0f2d12f20b07-etc-selinux\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.390746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.389779 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b39d0a10-298e-4d2b-8469-0194658c0466-multus-daemon-config\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.391515 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.390079 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-var-lib-openvswitch\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.391515 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.390114 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mmjp\" (UniqueName: \"kubernetes.io/projected/6a1be0cb-488e-4e55-855e-06abda43369b-kube-api-access-2mmjp\") pod \"node-resolver-r9fcz\" (UID: \"6a1be0cb-488e-4e55-855e-06abda43369b\") " pod="openshift-dns/node-resolver-r9fcz" Jul 06 21:12:58.391515 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.390142 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-cnibin\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.391515 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.390169 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg5jc\" (UniqueName: \"kubernetes.io/projected/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-kube-api-access-hg5jc\") pod \"network-metrics-daemon-xv8m9\" (UID: \"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150\") " pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:12:58.391515 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.390201 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x64m\" (UniqueName: \"kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m\") pod \"network-check-target-mzmmk\" (UID: \"b821193f-628f-4ed3-b87d-e96f3c441ff7\") " pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:12:58.391515 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.390228 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b39d0a10-298e-4d2b-8469-0194658c0466-cni-binary-copy\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.391515 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:58.390232 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:12:58.391515 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.390258 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-run-k8s-cni-cncf-io\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.391515 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:58.390320 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret podName:c8194ca3-5e42-4623-aabb-091c40445eaa nodeName:}" failed. No retries permitted until 2026-07-06 21:12:58.890297189 +0000 UTC m=+3.015278596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret") pod "global-pull-secret-syncer-xjshg" (UID: "c8194ca3-5e42-4623-aabb-091c40445eaa") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:12:58.396310 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.396279 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2fc778ca-01e3-41a8-87ae-9e5d28afcaad-agent-certs\") pod \"konnectivity-agent-ddkph\" (UID: \"2fc778ca-01e3-41a8-87ae-9e5d28afcaad\") " pod="kube-system/konnectivity-agent-ddkph" Jul 06 21:12:58.404096 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.403641 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtt2x\" (UniqueName: \"kubernetes.io/projected/b64ab87a-5226-488c-a68a-0f2d12f20b07-kube-api-access-qtt2x\") pod \"aws-ebs-csi-driver-node-fpf8w\" (UID: \"b64ab87a-5226-488c-a68a-0f2d12f20b07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.404229 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.404140 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rszm9\" (UniqueName: \"kubernetes.io/projected/b490d547-2ba8-44f0-a6e6-898846f59cc9-kube-api-access-rszm9\") pod \"node-ca-8w488\" (UID: \"b490d547-2ba8-44f0-a6e6-898846f59cc9\") " pod="openshift-image-registry/node-ca-8w488" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490597 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-sysctl-conf\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490641 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a1be0cb-488e-4e55-855e-06abda43369b-tmp-dir\") pod \"node-resolver-r9fcz\" (UID: \"6a1be0cb-488e-4e55-855e-06abda43369b\") " pod="openshift-dns/node-resolver-r9fcz" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490668 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-var-lib-cni-bin\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490694 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-run-multus-certs\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490727 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-run-netns\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490754 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-log-socket\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490780 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-multus-cni-dir\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490804 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-etc-kubernetes\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-os-release\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490861 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-run-openvswitch\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490886 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-run-ovn\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490903 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-run-multus-certs\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490969 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-run-netns\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490978 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-log-socket\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491007 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-run-netns\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491052 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-multus-cni-dir\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491054 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-var-lib-cni-bin\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.493669 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.490911 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-run-netns\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491099 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-var-lib-cni-multus\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491122 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-os-release\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491144 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-sysconfig\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491165 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-sysctl-conf\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491172 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-etc-kubernetes\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491173 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-systemd\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491205 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-cni-bin\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491221 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-run-ovn\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491230 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzzjb\" (UniqueName: \"kubernetes.io/projected/c6690861-f325-41b3-bf8c-5e43798ec3d0-kube-api-access-gzzjb\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491256 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6753e71a-7591-4dca-b1ba-236c3e51c791-iptables-alerter-script\") pod \"iptables-alerter-98smj\" (UID: \"6753e71a-7591-4dca-b1ba-236c3e51c791\") " pod="openshift-network-operator/iptables-alerter-98smj" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491266 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-run-openvswitch\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491277 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-system-cni-dir\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491302 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-system-cni-dir\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491315 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-systemd\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491328 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-cni-binary-copy\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491353 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-var-lib-cni-multus\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.494711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491356 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491381 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-node-log\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491397 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-sysconfig\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491421 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-os-release\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491448 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-multus-conf-dir\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491475 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-modprobe-d\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491502 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-kubernetes\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491514 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-system-cni-dir\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491527 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-run-ovn-kubernetes\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491492 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a1be0cb-488e-4e55-855e-06abda43369b-tmp-dir\") pod \"node-resolver-r9fcz\" (UID: \"6a1be0cb-488e-4e55-855e-06abda43369b\") " pod="openshift-dns/node-resolver-r9fcz" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491573 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqggq\" (UniqueName: \"kubernetes.io/projected/6753e71a-7591-4dca-b1ba-236c3e51c791-kube-api-access-xqggq\") pod \"iptables-alerter-98smj\" (UID: \"6753e71a-7591-4dca-b1ba-236c3e51c791\") " pod="openshift-network-operator/iptables-alerter-98smj" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491597 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-node-log\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491600 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-multus-socket-dir-parent\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491637 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b39d0a10-298e-4d2b-8469-0194658c0466-multus-daemon-config\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491668 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-var-lib-openvswitch\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491678 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-os-release\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491695 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mmjp\" (UniqueName: \"kubernetes.io/projected/6a1be0cb-488e-4e55-855e-06abda43369b-kube-api-access-2mmjp\") pod \"node-resolver-r9fcz\" (UID: \"6a1be0cb-488e-4e55-855e-06abda43369b\") " pod="openshift-dns/node-resolver-r9fcz" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491721 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-cnibin\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.495650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491725 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-multus-conf-dir\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491747 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg5jc\" (UniqueName: \"kubernetes.io/projected/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-kube-api-access-hg5jc\") pod \"network-metrics-daemon-xv8m9\" (UID: \"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150\") " pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491772 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x64m\" (UniqueName: \"kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m\") pod \"network-check-target-mzmmk\" (UID: \"b821193f-628f-4ed3-b87d-e96f3c441ff7\") " pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491794 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b39d0a10-298e-4d2b-8469-0194658c0466-cni-binary-copy\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491819 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-run-k8s-cni-cncf-io\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491826 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-modprobe-d\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491845 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-sysctl-d\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491870 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-run\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491878 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-kubernetes\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491893 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-sys\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491919 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-lib-modules\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491922 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-run-ovn-kubernetes\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491944 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs\") pod \"network-metrics-daemon-xv8m9\" (UID: \"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150\") " pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491967 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-var-lib-kubelet\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.491991 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492017 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwb8w\" (UniqueName: \"kubernetes.io/projected/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-kube-api-access-nwb8w\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492060 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-kubelet\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.496631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492086 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-etc-openvswitch\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492105 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6753e71a-7591-4dca-b1ba-236c3e51c791-iptables-alerter-script\") pod \"iptables-alerter-98smj\" (UID: \"6753e71a-7591-4dca-b1ba-236c3e51c791\") " pod="openshift-network-operator/iptables-alerter-98smj" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492114 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492144 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6690861-f325-41b3-bf8c-5e43798ec3d0-ovn-node-metrics-cert\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492226 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-cni-bin\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492227 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-system-cni-dir\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492288 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-multus-socket-dir-parent\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492466 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6kj4\" (UniqueName: \"kubernetes.io/projected/b39d0a10-298e-4d2b-8469-0194658c0466-kube-api-access-k6kj4\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492565 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/701babd3-9d0a-462d-b559-7508529a25f4-etc-tuned\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492594 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-systemd-units\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492621 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knrbw\" (UniqueName: \"kubernetes.io/projected/701babd3-9d0a-462d-b559-7508529a25f4-kube-api-access-knrbw\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492651 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-hostroot\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492676 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-run-systemd\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492712 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c6690861-f325-41b3-bf8c-5e43798ec3d0-ovnkube-script-lib\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492738 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-cnibin\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492763 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c6690861-f325-41b3-bf8c-5e43798ec3d0-ovnkube-config\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492789 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-slash\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.497485 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492823 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-cni-netd\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492841 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b39d0a10-298e-4d2b-8469-0194658c0466-multus-daemon-config\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-var-lib-kubelet\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492893 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-host\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492902 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-var-lib-kubelet\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492933 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/701babd3-9d0a-462d-b559-7508529a25f4-tmp\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492947 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-host\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492960 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a1be0cb-488e-4e55-855e-06abda43369b-hosts-file\") pod \"node-resolver-r9fcz\" (UID: \"6a1be0cb-488e-4e55-855e-06abda43369b\") " pod="openshift-dns/node-resolver-r9fcz" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492989 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-var-lib-openvswitch\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492988 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6753e71a-7591-4dca-b1ba-236c3e51c791-host-slash\") pod \"iptables-alerter-98smj\" (UID: \"6753e71a-7591-4dca-b1ba-236c3e51c791\") " pod="openshift-network-operator/iptables-alerter-98smj" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.493027 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6753e71a-7591-4dca-b1ba-236c3e51c791-host-slash\") pod \"iptables-alerter-98smj\" (UID: \"6753e71a-7591-4dca-b1ba-236c3e51c791\") " pod="openshift-network-operator/iptables-alerter-98smj" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.493079 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-cnibin\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.493282 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.493853 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b39d0a10-298e-4d2b-8469-0194658c0466-cni-binary-copy\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.493911 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-host-run-k8s-cni-cncf-io\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494038 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-etc-sysctl-d\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494091 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-run\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494124 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c6690861-f325-41b3-bf8c-5e43798ec3d0-ovnkube-config\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.498412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494138 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-sys\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494192 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-run-systemd\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:58.494233 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:58.494342 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs podName:54c01a9d-5eb8-4b7d-959a-f2a6e6a43150 nodeName:}" failed. No retries permitted until 2026-07-06 21:12:58.994322909 +0000 UTC m=+3.119304335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs") pod "network-metrics-daemon-xv8m9" (UID: "54c01a9d-5eb8-4b7d-959a-f2a6e6a43150") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494526 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-etc-openvswitch\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494533 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-kubelet\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494587 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-var-lib-kubelet\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494608 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.492841 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-cni-binary-copy\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494670 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-hostroot\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494684 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c6690861-f325-41b3-bf8c-5e43798ec3d0-ovnkube-script-lib\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494748 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b39d0a10-298e-4d2b-8469-0194658c0466-cnibin\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494771 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/701babd3-9d0a-462d-b559-7508529a25f4-lib-modules\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494942 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-slash\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.494988 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-host-cni-netd\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.493030 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.495032 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c6690861-f325-41b3-bf8c-5e43798ec3d0-env-overrides\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.499313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.495043 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a1be0cb-488e-4e55-855e-06abda43369b-hosts-file\") pod \"node-resolver-r9fcz\" (UID: \"6a1be0cb-488e-4e55-855e-06abda43369b\") " pod="openshift-dns/node-resolver-r9fcz" Jul 06 21:12:58.500322 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.495042 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.500322 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.495172 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.500322 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.495295 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c6690861-f325-41b3-bf8c-5e43798ec3d0-systemd-units\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.500322 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.495427 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c6690861-f325-41b3-bf8c-5e43798ec3d0-env-overrides\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.500322 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.497899 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/701babd3-9d0a-462d-b559-7508529a25f4-tmp\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.500322 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.498880 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-12.ec2.internal" event={"ID":"54f38d8f384b65c8b8874f57c20a13d4","Type":"ContainerStarted","Data":"15f1ee738918e2727f1a7d9e83cf594c71233cf750ee408846b6a9c3e8c5317a"} Jul 06 21:12:58.500322 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.499002 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/701babd3-9d0a-462d-b559-7508529a25f4-etc-tuned\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.501377 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.501048 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal" event={"ID":"4d463b2868e96d13a950b18acfec1156","Type":"ContainerStarted","Data":"8413dfa4ce3db50cd40fd9087b5b75c9b46c1cce694e98f18d7df6c2da98aa7d"} Jul 06 21:12:58.501469 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.501436 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6690861-f325-41b3-bf8c-5e43798ec3d0-ovn-node-metrics-cert\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.508876 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.508850 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqggq\" (UniqueName: \"kubernetes.io/projected/6753e71a-7591-4dca-b1ba-236c3e51c791-kube-api-access-xqggq\") pod \"iptables-alerter-98smj\" (UID: \"6753e71a-7591-4dca-b1ba-236c3e51c791\") " pod="openshift-network-operator/iptables-alerter-98smj" Jul 06 21:12:58.512164 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:58.512079 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:12:58.512164 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:58.512103 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:12:58.512164 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:58.512118 2577 projected.go:196] Error preparing data for projected volume kube-api-access-7x64m for pod openshift-network-diagnostics/network-check-target-mzmmk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:12:58.512761 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:58.512737 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m podName:b821193f-628f-4ed3-b87d-e96f3c441ff7 nodeName:}" failed. No retries permitted until 2026-07-06 21:12:59.012708592 +0000 UTC m=+3.137690016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7x64m" (UniqueName: "kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m") pod "network-check-target-mzmmk" (UID: "b821193f-628f-4ed3-b87d-e96f3c441ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:12:58.518505 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.518442 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6kj4\" (UniqueName: \"kubernetes.io/projected/b39d0a10-298e-4d2b-8469-0194658c0466-kube-api-access-k6kj4\") pod \"multus-h4886\" (UID: \"b39d0a10-298e-4d2b-8469-0194658c0466\") " pod="openshift-multus/multus-h4886" Jul 06 21:12:58.518629 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.518510 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knrbw\" (UniqueName: \"kubernetes.io/projected/701babd3-9d0a-462d-b559-7508529a25f4-kube-api-access-knrbw\") pod \"tuned-crh49\" (UID: \"701babd3-9d0a-462d-b559-7508529a25f4\") " pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.520005 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.519980 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwb8w\" (UniqueName: \"kubernetes.io/projected/d291c60c-4d03-4c4d-8284-0f3d918d5ad1-kube-api-access-nwb8w\") pod \"multus-additional-cni-plugins-7ds6m\" (UID: \"d291c60c-4d03-4c4d-8284-0f3d918d5ad1\") " pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.520672 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.520646 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzzjb\" (UniqueName: \"kubernetes.io/projected/c6690861-f325-41b3-bf8c-5e43798ec3d0-kube-api-access-gzzjb\") pod \"ovnkube-node-g64lt\" (UID: \"c6690861-f325-41b3-bf8c-5e43798ec3d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.521036 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.521002 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mmjp\" (UniqueName: \"kubernetes.io/projected/6a1be0cb-488e-4e55-855e-06abda43369b-kube-api-access-2mmjp\") pod \"node-resolver-r9fcz\" (UID: \"6a1be0cb-488e-4e55-855e-06abda43369b\") " pod="openshift-dns/node-resolver-r9fcz" Jul 06 21:12:58.522164 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.522144 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg5jc\" (UniqueName: \"kubernetes.io/projected/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-kube-api-access-hg5jc\") pod \"network-metrics-daemon-xv8m9\" (UID: \"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150\") " pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:12:58.534772 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.534746 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 21:12:58.589905 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.589758 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" Jul 06 21:12:58.600551 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:58.600516 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64ab87a_5226_488c_a68a_0f2d12f20b07.slice/crio-0e4b7834f038b245eceac1a36bd6622292c818bfb0e89712ce11352e9b306bbd WatchSource:0}: Error finding container 0e4b7834f038b245eceac1a36bd6622292c818bfb0e89712ce11352e9b306bbd: Status 404 returned error can't find the container with id 0e4b7834f038b245eceac1a36bd6622292c818bfb0e89712ce11352e9b306bbd Jul 06 21:12:58.602880 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.602817 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ddkph" Jul 06 21:12:58.612929 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.612863 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8w488" Jul 06 21:12:58.613042 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:58.612982 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fc778ca_01e3_41a8_87ae_9e5d28afcaad.slice/crio-e362976e53d99a4e43d3ccbe101624ee819674a830c04b1beff37e052d62b160 WatchSource:0}: Error finding container e362976e53d99a4e43d3ccbe101624ee819674a830c04b1beff37e052d62b160: Status 404 returned error can't find the container with id e362976e53d99a4e43d3ccbe101624ee819674a830c04b1beff37e052d62b160 Jul 06 21:12:58.620777 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.620753 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7ds6m" Jul 06 21:12:58.631479 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.631224 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:12:58.632486 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:58.632459 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd291c60c_4d03_4c4d_8284_0f3d918d5ad1.slice/crio-21b3d530f17c8a5f67401746e419af0424247ebd67ab4800eb77819e2e6410c3 WatchSource:0}: Error finding container 21b3d530f17c8a5f67401746e419af0424247ebd67ab4800eb77819e2e6410c3: Status 404 returned error can't find the container with id 21b3d530f17c8a5f67401746e419af0424247ebd67ab4800eb77819e2e6410c3 Jul 06 21:12:58.640752 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.640732 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r9fcz" Jul 06 21:12:58.650759 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.650730 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-98smj" Jul 06 21:12:58.661811 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.661778 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h4886" Jul 06 21:12:58.667780 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.667762 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-crh49" Jul 06 21:12:58.899256 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:58.899212 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:12:58.899440 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:58.899387 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:12:58.899541 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:58.899462 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret podName:c8194ca3-5e42-4623-aabb-091c40445eaa nodeName:}" failed. No retries permitted until 2026-07-06 21:12:59.899439168 +0000 UTC m=+4.024420593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret") pod "global-pull-secret-syncer-xjshg" (UID: "c8194ca3-5e42-4623-aabb-091c40445eaa") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:12:59.000304 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:59.000272 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs\") pod \"network-metrics-daemon-xv8m9\" (UID: \"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150\") " pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:12:59.000451 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:59.000430 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:12:59.000520 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:59.000508 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs podName:54c01a9d-5eb8-4b7d-959a-f2a6e6a43150 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:00.000490443 +0000 UTC m=+4.125471883 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs") pod "network-metrics-daemon-xv8m9" (UID: "54c01a9d-5eb8-4b7d-959a-f2a6e6a43150") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:12:59.101238 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:59.101198 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x64m\" (UniqueName: \"kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m\") pod \"network-check-target-mzmmk\" (UID: \"b821193f-628f-4ed3-b87d-e96f3c441ff7\") " pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:12:59.101446 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:59.101418 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:12:59.101446 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:59.101444 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:12:59.101594 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:59.101459 2577 projected.go:196] Error preparing data for projected volume kube-api-access-7x64m for pod openshift-network-diagnostics/network-check-target-mzmmk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:12:59.101594 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:59.101547 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m podName:b821193f-628f-4ed3-b87d-e96f3c441ff7 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:00.101526629 +0000 UTC m=+4.226508054 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7x64m" (UniqueName: "kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m") pod "network-check-target-mzmmk" (UID: "b821193f-628f-4ed3-b87d-e96f3c441ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:12:59.337923 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:59.337864 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 21:07:58 +0000 UTC" deadline="2028-01-24 01:20:56.109808949 +0000 UTC" Jul 06 21:12:59.337923 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:59.337918 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13588h7m56.771894431s" Jul 06 21:12:59.484000 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:59.483966 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:12:59.484152 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:59.484109 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:12:59.504528 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:59.504428 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7ds6m" event={"ID":"d291c60c-4d03-4c4d-8284-0f3d918d5ad1","Type":"ContainerStarted","Data":"21b3d530f17c8a5f67401746e419af0424247ebd67ab4800eb77819e2e6410c3"} Jul 06 21:12:59.506642 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:59.506607 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8w488" event={"ID":"b490d547-2ba8-44f0-a6e6-898846f59cc9","Type":"ContainerStarted","Data":"690edabe0b1d73bc76a609d8b770ca2deaec968ce07eb48547759f9743682de5"} Jul 06 21:12:59.508448 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:59.508393 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ddkph" event={"ID":"2fc778ca-01e3-41a8-87ae-9e5d28afcaad","Type":"ContainerStarted","Data":"e362976e53d99a4e43d3ccbe101624ee819674a830c04b1beff37e052d62b160"} Jul 06 21:12:59.510376 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:59.510286 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" event={"ID":"b64ab87a-5226-488c-a68a-0f2d12f20b07","Type":"ContainerStarted","Data":"0e4b7834f038b245eceac1a36bd6622292c818bfb0e89712ce11352e9b306bbd"} Jul 06 21:12:59.805167 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:59.805146 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod701babd3_9d0a_462d_b559_7508529a25f4.slice/crio-7dc1f1c62f150178abadf5d77019772a45538ddec2f9ecf61b9bcab3f8964333 WatchSource:0}: Error finding container 7dc1f1c62f150178abadf5d77019772a45538ddec2f9ecf61b9bcab3f8964333: Status 404 returned error can't find the container with id 7dc1f1c62f150178abadf5d77019772a45538ddec2f9ecf61b9bcab3f8964333 Jul 06 21:12:59.821835 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:59.821802 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb39d0a10_298e_4d2b_8469_0194658c0466.slice/crio-f1b03489218a408e6ef593abe16a822be116157a95dac8e0e35235873fd7c84a WatchSource:0}: Error finding container f1b03489218a408e6ef593abe16a822be116157a95dac8e0e35235873fd7c84a: Status 404 returned error can't find the container with id f1b03489218a408e6ef593abe16a822be116157a95dac8e0e35235873fd7c84a Jul 06 21:12:59.825360 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:59.825307 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6753e71a_7591_4dca_b1ba_236c3e51c791.slice/crio-a022300a9c75f1dbaba1d05f9ffba7f23b1310508f4d6818f46120f780bc3b59 WatchSource:0}: Error finding container a022300a9c75f1dbaba1d05f9ffba7f23b1310508f4d6818f46120f780bc3b59: Status 404 returned error can't find the container with id a022300a9c75f1dbaba1d05f9ffba7f23b1310508f4d6818f46120f780bc3b59 Jul 06 21:12:59.827571 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:12:59.827192 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a1be0cb_488e_4e55_855e_06abda43369b.slice/crio-3824a8838b7d5ed08e4815a9206cf17894cc14e1f61f66b47b9b3429e8e51fc1 WatchSource:0}: Error finding container 3824a8838b7d5ed08e4815a9206cf17894cc14e1f61f66b47b9b3429e8e51fc1: Status 404 returned error can't find the container with id 3824a8838b7d5ed08e4815a9206cf17894cc14e1f61f66b47b9b3429e8e51fc1 Jul 06 21:12:59.907482 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:12:59.907440 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:12:59.907637 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:59.907615 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:12:59.907702 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:12:59.907687 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret podName:c8194ca3-5e42-4623-aabb-091c40445eaa nodeName:}" failed. No retries permitted until 2026-07-06 21:13:01.907665282 +0000 UTC m=+6.032646685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret") pod "global-pull-secret-syncer-xjshg" (UID: "c8194ca3-5e42-4623-aabb-091c40445eaa") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:13:00.008821 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.008780 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs\") pod \"network-metrics-daemon-xv8m9\" (UID: \"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150\") " pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:00.009015 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:00.009004 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:13:00.009082 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:00.009070 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs podName:54c01a9d-5eb8-4b7d-959a-f2a6e6a43150 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:02.009050436 +0000 UTC m=+6.134031843 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs") pod "network-metrics-daemon-xv8m9" (UID: "54c01a9d-5eb8-4b7d-959a-f2a6e6a43150") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:13:00.109433 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.109195 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x64m\" (UniqueName: \"kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m\") pod \"network-check-target-mzmmk\" (UID: \"b821193f-628f-4ed3-b87d-e96f3c441ff7\") " pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:00.109643 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:00.109606 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:13:00.109643 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:00.109631 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:13:00.109643 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:00.109643 2577 projected.go:196] Error preparing data for projected volume kube-api-access-7x64m for pod openshift-network-diagnostics/network-check-target-mzmmk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:13:00.109852 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:00.109707 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m podName:b821193f-628f-4ed3-b87d-e96f3c441ff7 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:02.109687971 +0000 UTC m=+6.234669392 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7x64m" (UniqueName: "kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m") pod "network-check-target-mzmmk" (UID: "b821193f-628f-4ed3-b87d-e96f3c441ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:13:00.338189 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.338143 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 21:07:58 +0000 UTC" deadline="2027-12-25 05:52:11.565784713 +0000 UTC" Jul 06 21:13:00.338189 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.338187 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12872h39m11.227601855s" Jul 06 21:13:00.486321 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.485592 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:00.486321 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:00.485727 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:00.486804 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.486643 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:00.486804 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:00.486764 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:00.536790 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.536753 2577 generic.go:356] "Generic (PLEG): container finished" podID="4d463b2868e96d13a950b18acfec1156" containerID="9a1cbb0fa5d4ce10858a2280bb79fd83aa7c869e06ec0c2ad9d532d9b378abe9" exitCode=0 Jul 06 21:13:00.536962 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.536855 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal" event={"ID":"4d463b2868e96d13a950b18acfec1156","Type":"ContainerDied","Data":"9a1cbb0fa5d4ce10858a2280bb79fd83aa7c869e06ec0c2ad9d532d9b378abe9"} Jul 06 21:13:00.541332 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.540646 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-12.ec2.internal" event={"ID":"54f38d8f384b65c8b8874f57c20a13d4","Type":"ContainerStarted","Data":"512ba726127a130a1925446050ac6f7c36e37df987f159ac8c47e2c0c30f72ce"} Jul 06 21:13:00.551271 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.551239 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h4886" event={"ID":"b39d0a10-298e-4d2b-8469-0194658c0466","Type":"ContainerStarted","Data":"f1b03489218a408e6ef593abe16a822be116157a95dac8e0e35235873fd7c84a"} Jul 06 21:13:00.568027 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.567989 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" event={"ID":"c6690861-f325-41b3-bf8c-5e43798ec3d0","Type":"ContainerStarted","Data":"0309b7ba0dba48aad0c6d67f93479a8341486a00ddddc3bac3d794fac8e1b635"} Jul 06 21:13:00.583781 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.583751 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r9fcz" event={"ID":"6a1be0cb-488e-4e55-855e-06abda43369b","Type":"ContainerStarted","Data":"3824a8838b7d5ed08e4815a9206cf17894cc14e1f61f66b47b9b3429e8e51fc1"} Jul 06 21:13:00.588808 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.588776 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-98smj" event={"ID":"6753e71a-7591-4dca-b1ba-236c3e51c791","Type":"ContainerStarted","Data":"a022300a9c75f1dbaba1d05f9ffba7f23b1310508f4d6818f46120f780bc3b59"} Jul 06 21:13:00.594100 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:00.594070 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-crh49" event={"ID":"701babd3-9d0a-462d-b559-7508529a25f4","Type":"ContainerStarted","Data":"7dc1f1c62f150178abadf5d77019772a45538ddec2f9ecf61b9bcab3f8964333"} Jul 06 21:13:01.484262 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:01.483731 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:01.484262 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:01.483866 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:01.925511 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:01.924883 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:01.925511 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:01.925068 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:13:01.925511 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:01.925130 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret podName:c8194ca3-5e42-4623-aabb-091c40445eaa nodeName:}" failed. No retries permitted until 2026-07-06 21:13:05.925111132 +0000 UTC m=+10.050092543 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret") pod "global-pull-secret-syncer-xjshg" (UID: "c8194ca3-5e42-4623-aabb-091c40445eaa") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:13:02.026396 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:02.025847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs\") pod \"network-metrics-daemon-xv8m9\" (UID: \"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150\") " pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:02.026396 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:02.025999 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:13:02.026396 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:02.026059 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs podName:54c01a9d-5eb8-4b7d-959a-f2a6e6a43150 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:06.026041724 +0000 UTC m=+10.151023130 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs") pod "network-metrics-daemon-xv8m9" (UID: "54c01a9d-5eb8-4b7d-959a-f2a6e6a43150") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:13:02.127149 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:02.127046 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x64m\" (UniqueName: \"kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m\") pod \"network-check-target-mzmmk\" (UID: \"b821193f-628f-4ed3-b87d-e96f3c441ff7\") " pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:02.127332 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:02.127238 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:13:02.127332 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:02.127263 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:13:02.127332 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:02.127277 2577 projected.go:196] Error preparing data for projected volume kube-api-access-7x64m for pod openshift-network-diagnostics/network-check-target-mzmmk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:13:02.127509 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:02.127337 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m podName:b821193f-628f-4ed3-b87d-e96f3c441ff7 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:06.127318031 +0000 UTC m=+10.252299447 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7x64m" (UniqueName: "kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m") pod "network-check-target-mzmmk" (UID: "b821193f-628f-4ed3-b87d-e96f3c441ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:13:02.485061 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:02.484559 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:02.485061 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:02.484692 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:02.485839 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:02.485672 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:02.485839 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:02.485793 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:03.484374 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:03.484339 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:03.484578 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:03.484490 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:04.484480 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:04.484427 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:04.484920 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:04.484432 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:04.484920 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:04.484637 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:04.484920 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:04.484758 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:05.484053 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:05.484023 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:05.484224 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:05.484154 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:05.959900 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:05.959821 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:05.960373 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:05.959991 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:13:05.960373 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:05.960061 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret podName:c8194ca3-5e42-4623-aabb-091c40445eaa nodeName:}" failed. No retries permitted until 2026-07-06 21:13:13.960042114 +0000 UTC m=+18.085023520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret") pod "global-pull-secret-syncer-xjshg" (UID: "c8194ca3-5e42-4623-aabb-091c40445eaa") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:13:06.060792 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:06.060750 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs\") pod \"network-metrics-daemon-xv8m9\" (UID: \"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150\") " pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:06.060968 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:06.060892 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:13:06.061027 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:06.060977 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs podName:54c01a9d-5eb8-4b7d-959a-f2a6e6a43150 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:14.060954988 +0000 UTC m=+18.185936400 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs") pod "network-metrics-daemon-xv8m9" (UID: "54c01a9d-5eb8-4b7d-959a-f2a6e6a43150") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:13:06.162053 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:06.161997 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x64m\" (UniqueName: \"kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m\") pod \"network-check-target-mzmmk\" (UID: \"b821193f-628f-4ed3-b87d-e96f3c441ff7\") " pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:06.162244 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:06.162190 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:13:06.162244 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:06.162217 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:13:06.162244 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:06.162231 2577 projected.go:196] Error preparing data for projected volume kube-api-access-7x64m for pod openshift-network-diagnostics/network-check-target-mzmmk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:13:06.162395 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:06.162292 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m podName:b821193f-628f-4ed3-b87d-e96f3c441ff7 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:14.162273106 +0000 UTC m=+18.287254518 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7x64m" (UniqueName: "kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m") pod "network-check-target-mzmmk" (UID: "b821193f-628f-4ed3-b87d-e96f3c441ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:13:06.484073 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:06.484042 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:06.484238 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:06.484199 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:06.484699 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:06.484677 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:06.484812 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:06.484793 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:07.484394 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:07.484361 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:07.484943 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:07.484506 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:08.484050 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:08.484007 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:08.484201 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:08.484128 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:08.484201 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:08.484177 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:08.484358 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:08.484297 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:09.484760 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:09.484720 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:09.485219 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:09.484846 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:10.484520 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:10.484479 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:10.484710 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:10.484526 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:10.484710 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:10.484637 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:10.484786 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:10.484729 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:11.484038 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:11.484000 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:11.484206 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:11.484122 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:12.484227 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:12.484191 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:12.484607 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:12.484205 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:12.484607 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:12.484332 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:12.484607 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:12.484379 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:13.484567 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:13.484534 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:13.484981 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:13.484670 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:14.021376 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:14.021326 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:14.021543 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:14.021524 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:13:14.021621 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:14.021609 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret podName:c8194ca3-5e42-4623-aabb-091c40445eaa nodeName:}" failed. No retries permitted until 2026-07-06 21:13:30.02158792 +0000 UTC m=+34.146569325 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret") pod "global-pull-secret-syncer-xjshg" (UID: "c8194ca3-5e42-4623-aabb-091c40445eaa") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:13:14.121811 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:14.121771 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs\") pod \"network-metrics-daemon-xv8m9\" (UID: \"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150\") " pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:14.121980 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:14.121896 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:13:14.121980 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:14.121962 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs podName:54c01a9d-5eb8-4b7d-959a-f2a6e6a43150 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:30.121946251 +0000 UTC m=+34.246927652 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs") pod "network-metrics-daemon-xv8m9" (UID: "54c01a9d-5eb8-4b7d-959a-f2a6e6a43150") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:13:14.222541 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:14.222503 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x64m\" (UniqueName: \"kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m\") pod \"network-check-target-mzmmk\" (UID: \"b821193f-628f-4ed3-b87d-e96f3c441ff7\") " pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:14.222688 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:14.222674 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:13:14.222723 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:14.222694 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:13:14.222723 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:14.222705 2577 projected.go:196] Error preparing data for projected volume kube-api-access-7x64m for pod openshift-network-diagnostics/network-check-target-mzmmk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:13:14.222803 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:14.222759 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m podName:b821193f-628f-4ed3-b87d-e96f3c441ff7 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:30.222740782 +0000 UTC m=+34.347722183 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7x64m" (UniqueName: "kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m") pod "network-check-target-mzmmk" (UID: "b821193f-628f-4ed3-b87d-e96f3c441ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:13:14.488080 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:14.488048 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:14.488554 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:14.488181 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:14.488554 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:14.488253 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:14.488554 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:14.488363 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:15.484124 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:15.484080 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:15.484311 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:15.484215 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:16.485137 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:16.485093 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:16.485625 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:16.485224 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:16.485625 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:16.485281 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:16.485625 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:16.485380 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:17.483924 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:17.483893 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:17.484041 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:17.484019 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:17.632850 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:17.632815 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal" event={"ID":"4d463b2868e96d13a950b18acfec1156","Type":"ContainerStarted","Data":"2470b8022c9627496861c77aec2b6638aa2972f8d130be4346c114dc129cbd90"} Jul 06 21:13:17.634456 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:17.634384 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7ds6m" event={"ID":"d291c60c-4d03-4c4d-8284-0f3d918d5ad1","Type":"ContainerStarted","Data":"663e8f4d5f720f917f00b08bfd1776143eae9f7a6949e1a9e37b6809a7dcdb72"} Jul 06 21:13:17.635895 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:17.635872 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8w488" event={"ID":"b490d547-2ba8-44f0-a6e6-898846f59cc9","Type":"ContainerStarted","Data":"1af5fc5b7546f757ed255c68b9b07f7afd63b0c108d9c322ddfd4fb93267f4f3"} Jul 06 21:13:17.637372 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:17.637308 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ddkph" event={"ID":"2fc778ca-01e3-41a8-87ae-9e5d28afcaad","Type":"ContainerStarted","Data":"49429620ca506e8959a429add7708157bf68d5ac638d03c9b1e15a9e612f44f9"} Jul 06 21:13:17.640362 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:17.640339 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r9fcz" event={"ID":"6a1be0cb-488e-4e55-855e-06abda43369b","Type":"ContainerStarted","Data":"3e0037f55cbc8eff0e98b9d9306289296c28b18f42f5e642c5ebeb47f9100689"} Jul 06 21:13:17.650634 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:17.650402 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-12.ec2.internal" podStartSLOduration=20.650385309 podStartE2EDuration="20.650385309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:12:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:00.566139006 +0000 UTC m=+4.691120485" watchObservedRunningTime="2026-07-06 21:13:17.650385309 +0000 UTC m=+21.775366732" Jul 06 21:13:17.650718 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:17.650678 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-12.ec2.internal" podStartSLOduration=20.650671554 podStartE2EDuration="20.650671554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:12:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:17.649877916 +0000 UTC m=+21.774859350" watchObservedRunningTime="2026-07-06 21:13:17.650671554 +0000 UTC m=+21.775652977" Jul 06 21:13:17.690582 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:17.690525 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ddkph" podStartSLOduration=2.9437878 podStartE2EDuration="21.690502276s" totalImagesPullingTime="18.746714476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:17.690349448 +0000 UTC m=+21.815330871" watchObservedRunningTime="2026-07-06 21:13:17.690502276 +0000 UTC m=+21.815483702" Jul 06 21:13:17.705269 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:17.705224 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8w488" podStartSLOduration=11.311508547 podStartE2EDuration="21.705208319s" totalImagesPullingTime="10.393699772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:17.704601331 +0000 UTC m=+21.829582756" watchObservedRunningTime="2026-07-06 21:13:17.705208319 +0000 UTC m=+21.830189741" Jul 06 21:13:17.718906 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:17.718856 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-r9fcz" podStartSLOduration=4.18731563 podStartE2EDuration="21.718841845s" totalImagesPullingTime="17.531526215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:17.718721235 +0000 UTC m=+21.843702658" watchObservedRunningTime="2026-07-06 21:13:17.718841845 +0000 UTC m=+21.843823267" Jul 06 21:13:18.484799 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.484577 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:18.484942 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.484577 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:18.484942 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:18.484858 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:18.484942 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:18.484906 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:18.643975 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.643948 2577 generic.go:356] "Generic (PLEG): container finished" podID="d291c60c-4d03-4c4d-8284-0f3d918d5ad1" containerID="663e8f4d5f720f917f00b08bfd1776143eae9f7a6949e1a9e37b6809a7dcdb72" exitCode=0 Jul 06 21:13:18.644692 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.644020 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7ds6m" event={"ID":"d291c60c-4d03-4c4d-8284-0f3d918d5ad1","Type":"ContainerDied","Data":"663e8f4d5f720f917f00b08bfd1776143eae9f7a6949e1a9e37b6809a7dcdb72"} Jul 06 21:13:18.645489 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.645395 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h4886" event={"ID":"b39d0a10-298e-4d2b-8469-0194658c0466","Type":"ContainerStarted","Data":"6be9c3c81f1869cd2c5f63dd6ee97aaaffe55fe425e2ef09ac7adbb7719a3470"} Jul 06 21:13:18.647665 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.647649 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:13:18.647935 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.647916 2577 generic.go:356] "Generic (PLEG): container finished" podID="c6690861-f325-41b3-bf8c-5e43798ec3d0" containerID="10279fb13f1346e53c3b4e1f0e1aae3486c99273815c2dff414701dace37650f" exitCode=1 Jul 06 21:13:18.647999 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.647983 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" event={"ID":"c6690861-f325-41b3-bf8c-5e43798ec3d0","Type":"ContainerStarted","Data":"1ecc44f816030411b9ceb08fa4ea758bf937702abc43f3f6c73a3461d911890d"} Jul 06 21:13:18.648051 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.648039 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" event={"ID":"c6690861-f325-41b3-bf8c-5e43798ec3d0","Type":"ContainerStarted","Data":"c38d5e2dafdc0995d08a4133ba41b29e88fd9fbee05c02cfe87ed356b33a04af"} Jul 06 21:13:18.648087 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.648081 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" event={"ID":"c6690861-f325-41b3-bf8c-5e43798ec3d0","Type":"ContainerDied","Data":"10279fb13f1346e53c3b4e1f0e1aae3486c99273815c2dff414701dace37650f"} Jul 06 21:13:18.648127 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.648111 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" event={"ID":"c6690861-f325-41b3-bf8c-5e43798ec3d0","Type":"ContainerStarted","Data":"dbf24b89419cba1db472d379b88eeb4109b91cc60bc8beef4d9fd2e3866459fb"} Jul 06 21:13:18.649292 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.649268 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" event={"ID":"b64ab87a-5226-488c-a68a-0f2d12f20b07","Type":"ContainerStarted","Data":"78b8d80cd0eb32767339088b5c85d7fd3e811e9e9419a3b94eb13914d86333d3"} Jul 06 21:13:18.650673 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.650642 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-crh49" event={"ID":"701babd3-9d0a-462d-b559-7508529a25f4","Type":"ContainerStarted","Data":"df20a149a1618c4c3ad4bc58bcba980101365506135b444d52ee9a8fe94daf68"} Jul 06 21:13:18.698105 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.698070 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-crh49" podStartSLOduration=5.151515423 podStartE2EDuration="22.698056234s" totalImagesPullingTime="17.546540811s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:18.682225055 +0000 UTC m=+22.807206478" watchObservedRunningTime="2026-07-06 21:13:18.698056234 +0000 UTC m=+22.823037657" Jul 06 21:13:18.698395 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.698377 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h4886" podStartSLOduration=5.09321067 podStartE2EDuration="22.698370425s" totalImagesPullingTime="17.605159755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:18.697976576 +0000 UTC m=+22.822957999" watchObservedRunningTime="2026-07-06 21:13:18.698370425 +0000 UTC m=+22.823351848" Jul 06 21:13:18.970858 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:18.970832 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 21:13:19.402918 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:19.402797 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T21:13:18.970852615Z","UUID":"2600c69c-50d2-43d5-aaee-0a6be4a0582d","Handler":null,"Name":"","Endpoint":""} Jul 06 21:13:19.404655 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:19.404628 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 21:13:19.404791 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:19.404669 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 21:13:19.483894 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:19.483864 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:19.484069 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:19.483977 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:19.654949 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:19.654877 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" event={"ID":"b64ab87a-5226-488c-a68a-0f2d12f20b07","Type":"ContainerStarted","Data":"64d975288294e6dc052e66291f6c23c51f1c59a7a06602dee59b4f36982cf9d1"} Jul 06 21:13:19.656239 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:19.656216 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-98smj" event={"ID":"6753e71a-7591-4dca-b1ba-236c3e51c791","Type":"ContainerStarted","Data":"e9a2f9cafd84697a3fb7eb2906066ba15a1aa9b40f3c125372bf7a37e5302774"} Jul 06 21:13:19.658893 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:19.658874 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:13:19.659206 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:19.659182 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" event={"ID":"c6690861-f325-41b3-bf8c-5e43798ec3d0","Type":"ContainerStarted","Data":"747579c520744c92e6ef9aa06b5c6c9019ce2f81af7416cd658bfeb482215b11"} Jul 06 21:13:19.659278 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:19.659263 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" event={"ID":"c6690861-f325-41b3-bf8c-5e43798ec3d0","Type":"ContainerStarted","Data":"34afa112f63cc52a3e62fefafc638effd3e021a3f20ab896f6e70ebe15584b8b"} Jul 06 21:13:19.670874 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:19.670833 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-98smj" podStartSLOduration=6.136133281 podStartE2EDuration="23.670820574s" totalImagesPullingTime="17.534687293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:19.670451795 +0000 UTC m=+23.795433220" watchObservedRunningTime="2026-07-06 21:13:19.670820574 +0000 UTC m=+23.795801997" Jul 06 21:13:20.487495 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:20.487403 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:20.487495 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:20.487439 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:20.487751 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:20.487568 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:20.487751 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:20.487698 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:20.663191 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:20.663160 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" event={"ID":"b64ab87a-5226-488c-a68a-0f2d12f20b07","Type":"ContainerStarted","Data":"6f75644e745bc407a7ad5e19a1371c25aea4b9138fd8ff10ce66926b4c6005da"} Jul 06 21:13:20.681268 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:20.681224 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpf8w" podStartSLOduration=3.546508742 podStartE2EDuration="24.681209748s" totalImagesPullingTime="21.134701006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:12:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:20.680861512 +0000 UTC m=+24.805842935" watchObservedRunningTime="2026-07-06 21:13:20.681209748 +0000 UTC m=+24.806191170" Jul 06 21:13:21.484624 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:21.484427 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:21.484846 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:21.484713 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:21.668391 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:21.668361 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:13:21.668789 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:21.668753 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" event={"ID":"c6690861-f325-41b3-bf8c-5e43798ec3d0","Type":"ContainerStarted","Data":"ee1a21fe958e8925580d0e06ff86dc6c21041826c0f51f2c0ab291bdaf4b51d0"} Jul 06 21:13:22.240879 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:22.240835 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ddkph" Jul 06 21:13:22.241529 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:22.241505 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ddkph" Jul 06 21:13:22.488360 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:22.488331 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:22.488530 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:22.488373 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:22.488530 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:22.488484 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:22.488628 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:22.488597 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:22.671731 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:22.671697 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ddkph" Jul 06 21:13:22.672286 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:22.672264 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ddkph" Jul 06 21:13:23.483937 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:23.483898 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:23.484148 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:23.484042 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:24.484719 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:24.484533 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:24.485293 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:24.484537 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:24.485293 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:24.484810 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:24.485293 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:24.484865 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:24.677524 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:24.677488 2577 generic.go:356] "Generic (PLEG): container finished" podID="d291c60c-4d03-4c4d-8284-0f3d918d5ad1" containerID="cac694a410f6134d3c179f7369b38f61ccc2d6a51687eca20a16821aced998b6" exitCode=0 Jul 06 21:13:24.677823 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:24.677574 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7ds6m" event={"ID":"d291c60c-4d03-4c4d-8284-0f3d918d5ad1","Type":"ContainerDied","Data":"cac694a410f6134d3c179f7369b38f61ccc2d6a51687eca20a16821aced998b6"} Jul 06 21:13:24.681196 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:24.681178 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:13:24.681638 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:24.681614 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" event={"ID":"c6690861-f325-41b3-bf8c-5e43798ec3d0","Type":"ContainerStarted","Data":"ec9ef2ae290d90c60196c17daa05a5937e11ed293f3e39b11f0a6da76f6791c0"} Jul 06 21:13:24.681959 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:24.681945 2577 scope.go:117] "RemoveContainer" containerID="10279fb13f1346e53c3b4e1f0e1aae3486c99273815c2dff414701dace37650f" Jul 06 21:13:25.484064 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.484029 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:25.484302 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:25.484132 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:25.685591 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.685561 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7ds6m" event={"ID":"d291c60c-4d03-4c4d-8284-0f3d918d5ad1","Type":"ContainerStarted","Data":"61ef38729ff3a24173e928d748c4c24b40881e42078f74338baf3b808d0bad45"} Jul 06 21:13:25.689910 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.689889 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:13:25.690315 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.690294 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" event={"ID":"c6690861-f325-41b3-bf8c-5e43798ec3d0","Type":"ContainerStarted","Data":"3da8be00340907bee4b3a309d2663b8252a5bf4b85809b6209d03593340243a3"} Jul 06 21:13:25.690646 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.690626 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:13:25.690743 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.690687 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:13:25.690743 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.690733 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:13:25.709031 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.709005 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:13:25.715010 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.714988 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:13:25.761148 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.760931 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" podStartSLOduration=11.595214742 podStartE2EDuration="29.760914825s" totalImagesPullingTime="18.165700083s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:25.759718856 +0000 UTC m=+29.884700291" watchObservedRunningTime="2026-07-06 21:13:25.760914825 +0000 UTC m=+29.885896247" Jul 06 21:13:25.798600 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.798570 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xjshg"] Jul 06 21:13:25.798769 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.798681 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:25.798769 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:25.798759 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:25.801626 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.801604 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xv8m9"] Jul 06 21:13:25.801724 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.801702 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:25.801808 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:25.801790 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:25.817030 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.817004 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mzmmk"] Jul 06 21:13:25.817151 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:25.817086 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:25.817205 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:25.817162 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:26.697593 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:26.697557 2577 generic.go:356] "Generic (PLEG): container finished" podID="d291c60c-4d03-4c4d-8284-0f3d918d5ad1" containerID="61ef38729ff3a24173e928d748c4c24b40881e42078f74338baf3b808d0bad45" exitCode=0 Jul 06 21:13:26.698032 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:26.697641 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7ds6m" event={"ID":"d291c60c-4d03-4c4d-8284-0f3d918d5ad1","Type":"ContainerDied","Data":"61ef38729ff3a24173e928d748c4c24b40881e42078f74338baf3b808d0bad45"} Jul 06 21:13:27.484222 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:27.484142 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:27.484222 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:27.484169 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:27.484222 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:27.484172 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:27.484482 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:27.484280 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:27.484482 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:27.484419 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:27.484582 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:27.484525 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:27.702044 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:27.702015 2577 generic.go:356] "Generic (PLEG): container finished" podID="d291c60c-4d03-4c4d-8284-0f3d918d5ad1" containerID="3cbafda38f9e3db239ad8d26c82dba2454859f80687a88a411507d545bb0d8be" exitCode=0 Jul 06 21:13:27.702651 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:27.702090 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7ds6m" event={"ID":"d291c60c-4d03-4c4d-8284-0f3d918d5ad1","Type":"ContainerDied","Data":"3cbafda38f9e3db239ad8d26c82dba2454859f80687a88a411507d545bb0d8be"} Jul 06 21:13:29.483997 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:29.483927 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:29.483997 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:29.483927 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:29.483997 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:29.483978 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:29.484674 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:29.484081 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjshg" podUID="c8194ca3-5e42-4623-aabb-091c40445eaa" Jul 06 21:13:29.484674 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:29.484131 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzmmk" podUID="b821193f-628f-4ed3-b87d-e96f3c441ff7" Jul 06 21:13:29.484674 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:29.484243 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xv8m9" podUID="54c01a9d-5eb8-4b7d-959a-f2a6e6a43150" Jul 06 21:13:30.043971 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.043932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:30.044151 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:30.044106 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:13:30.044201 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:30.044185 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret podName:c8194ca3-5e42-4623-aabb-091c40445eaa nodeName:}" failed. No retries permitted until 2026-07-06 21:14:02.044162972 +0000 UTC m=+66.169144396 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret") pod "global-pull-secret-syncer-xjshg" (UID: "c8194ca3-5e42-4623-aabb-091c40445eaa") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:13:30.144776 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.144743 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs\") pod \"network-metrics-daemon-xv8m9\" (UID: \"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150\") " pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:30.144948 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:30.144886 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:13:30.145008 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:30.144949 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs podName:54c01a9d-5eb8-4b7d-959a-f2a6e6a43150 nodeName:}" failed. No retries permitted until 2026-07-06 21:14:02.144930689 +0000 UTC m=+66.269912091 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs") pod "network-metrics-daemon-xv8m9" (UID: "54c01a9d-5eb8-4b7d-959a-f2a6e6a43150") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:13:30.245622 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.245589 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x64m\" (UniqueName: \"kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m\") pod \"network-check-target-mzmmk\" (UID: \"b821193f-628f-4ed3-b87d-e96f3c441ff7\") " pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:30.245812 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:30.245783 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:13:30.245812 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:30.245806 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:13:30.245922 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:30.245818 2577 projected.go:196] Error preparing data for projected volume kube-api-access-7x64m for pod openshift-network-diagnostics/network-check-target-mzmmk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:13:30.245922 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:30.245881 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m podName:b821193f-628f-4ed3-b87d-e96f3c441ff7 nodeName:}" failed. No retries permitted until 2026-07-06 21:14:02.245862938 +0000 UTC m=+66.370844366 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7x64m" (UniqueName: "kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m") pod "network-check-target-mzmmk" (UID: "b821193f-628f-4ed3-b87d-e96f3c441ff7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:13:30.628710 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.628674 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-12.ec2.internal" event="NodeReady" Jul 06 21:13:30.629112 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.628813 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 21:13:30.668832 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.668588 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5dddcc6c5c-nmgkt"] Jul 06 21:13:30.670791 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.670765 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.673928 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.673908 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 21:13:30.674058 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.674035 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 21:13:30.674435 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.674328 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 21:13:30.674595 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.674574 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dgf9v\"" Jul 06 21:13:30.678778 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.678757 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 21:13:30.679873 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.679442 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-w8w2r"] Jul 06 21:13:30.681388 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.681371 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:30.683723 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.683705 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 21:13:30.683822 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.683805 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4w8fd\"" Jul 06 21:13:30.683902 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.683724 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 21:13:30.690871 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.690608 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dwg5d"] Jul 06 21:13:30.694197 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.694175 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dddcc6c5c-nmgkt"] Jul 06 21:13:30.694309 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.694213 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w8w2r"] Jul 06 21:13:30.694495 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.694478 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dwg5d" Jul 06 21:13:30.697454 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.697436 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 21:13:30.697741 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.697724 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 21:13:30.697898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.697868 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kccmm\"" Jul 06 21:13:30.698013 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.697997 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 21:13:30.702752 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.702729 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dwg5d"] Jul 06 21:13:30.835509 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.835460 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:30.850081 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850052 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de552445-923f-48e7-9a47-60f066827b50-tmp-dir\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:30.850225 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850088 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-bound-sa-token\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.850225 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850109 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de552445-923f-48e7-9a47-60f066827b50-config-volume\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:30.850225 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5bcf19dc-706f-4379-a0a1-18029d327224-trusted-ca\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.850225 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850218 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78439b85-9901-45a4-898d-dbeb91d2eacc-cert\") pod \"ingress-canary-dwg5d\" (UID: \"78439b85-9901-45a4-898d-dbeb91d2eacc\") " pod="openshift-ingress-canary/ingress-canary-dwg5d" Jul 06 21:13:30.850449 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850246 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5bcf19dc-706f-4379-a0a1-18029d327224-registry-certificates\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.850449 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850287 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxc9t\" (UniqueName: \"kubernetes.io/projected/78439b85-9901-45a4-898d-dbeb91d2eacc-kube-api-access-jxc9t\") pod \"ingress-canary-dwg5d\" (UID: \"78439b85-9901-45a4-898d-dbeb91d2eacc\") " pod="openshift-ingress-canary/ingress-canary-dwg5d" Jul 06 21:13:30.850449 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850311 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5bcf19dc-706f-4379-a0a1-18029d327224-image-registry-private-configuration\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.850449 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850336 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5bcf19dc-706f-4379-a0a1-18029d327224-ca-trust-extracted\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.850449 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850359 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:30.850679 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850479 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gltht\" (UniqueName: \"kubernetes.io/projected/de552445-923f-48e7-9a47-60f066827b50-kube-api-access-gltht\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:30.850679 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850517 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnb74\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-kube-api-access-hnb74\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.850679 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850545 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.850679 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.850570 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5bcf19dc-706f-4379-a0a1-18029d327224-installation-pull-secrets\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.945331 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.945247 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:30.951121 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951090 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxc9t\" (UniqueName: \"kubernetes.io/projected/78439b85-9901-45a4-898d-dbeb91d2eacc-kube-api-access-jxc9t\") pod \"ingress-canary-dwg5d\" (UID: \"78439b85-9901-45a4-898d-dbeb91d2eacc\") " pod="openshift-ingress-canary/ingress-canary-dwg5d" Jul 06 21:13:30.951251 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951128 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5bcf19dc-706f-4379-a0a1-18029d327224-image-registry-private-configuration\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.951341 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951316 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5bcf19dc-706f-4379-a0a1-18029d327224-ca-trust-extracted\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.951399 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951368 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:30.951483 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951446 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gltht\" (UniqueName: \"kubernetes.io/projected/de552445-923f-48e7-9a47-60f066827b50-kube-api-access-gltht\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:30.951555 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951495 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnb74\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-kube-api-access-hnb74\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.951614 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:30.951573 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 21:13:30.951664 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:30.951643 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls podName:de552445-923f-48e7-9a47-60f066827b50 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:31.45162236 +0000 UTC m=+35.576603769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls") pod "dns-default-w8w2r" (UID: "de552445-923f-48e7-9a47-60f066827b50") : secret "dns-default-metrics-tls" not found Jul 06 21:13:30.951726 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951669 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.951779 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5bcf19dc-706f-4379-a0a1-18029d327224-installation-pull-secrets\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.951779 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951755 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de552445-923f-48e7-9a47-60f066827b50-tmp-dir\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:30.951779 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951760 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5bcf19dc-706f-4379-a0a1-18029d327224-ca-trust-extracted\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.951921 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951789 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-bound-sa-token\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.951921 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951825 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de552445-923f-48e7-9a47-60f066827b50-config-volume\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:30.951921 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:30.951835 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 21:13:30.951921 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951848 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5bcf19dc-706f-4379-a0a1-18029d327224-trusted-ca\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.951921 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951904 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78439b85-9901-45a4-898d-dbeb91d2eacc-cert\") pod \"ingress-canary-dwg5d\" (UID: \"78439b85-9901-45a4-898d-dbeb91d2eacc\") " pod="openshift-ingress-canary/ingress-canary-dwg5d" Jul 06 21:13:30.952147 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.951932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5bcf19dc-706f-4379-a0a1-18029d327224-registry-certificates\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.952147 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.952045 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de552445-923f-48e7-9a47-60f066827b50-tmp-dir\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:30.952147 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:30.951853 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dddcc6c5c-nmgkt: secret "image-registry-tls" not found Jul 06 21:13:30.952285 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:30.952189 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls podName:5bcf19dc-706f-4379-a0a1-18029d327224 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:31.45216727 +0000 UTC m=+35.577148692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls") pod "image-registry-5dddcc6c5c-nmgkt" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224") : secret "image-registry-tls" not found Jul 06 21:13:30.952925 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.952426 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de552445-923f-48e7-9a47-60f066827b50-config-volume\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:30.952925 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.952482 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5bcf19dc-706f-4379-a0a1-18029d327224-registry-certificates\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.952925 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.952811 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5bcf19dc-706f-4379-a0a1-18029d327224-trusted-ca\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.956193 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.956128 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5bcf19dc-706f-4379-a0a1-18029d327224-installation-pull-secrets\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.956193 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.956159 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78439b85-9901-45a4-898d-dbeb91d2eacc-cert\") pod \"ingress-canary-dwg5d\" (UID: \"78439b85-9901-45a4-898d-dbeb91d2eacc\") " pod="openshift-ingress-canary/ingress-canary-dwg5d" Jul 06 21:13:30.956193 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.956176 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5bcf19dc-706f-4379-a0a1-18029d327224-image-registry-private-configuration\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.960585 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.960505 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gltht\" (UniqueName: \"kubernetes.io/projected/de552445-923f-48e7-9a47-60f066827b50-kube-api-access-gltht\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:30.960705 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.960687 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxc9t\" (UniqueName: \"kubernetes.io/projected/78439b85-9901-45a4-898d-dbeb91d2eacc-kube-api-access-jxc9t\") pod \"ingress-canary-dwg5d\" (UID: \"78439b85-9901-45a4-898d-dbeb91d2eacc\") " pod="openshift-ingress-canary/ingress-canary-dwg5d" Jul 06 21:13:30.961625 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.961587 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnb74\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-kube-api-access-hnb74\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:30.961787 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:30.961750 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-bound-sa-token\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:31.004533 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.004498 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dwg5d" Jul 06 21:13:31.102635 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.102593 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:31.199870 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.199727 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:31.456434 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.456353 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:31.456434 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.456427 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:31.456661 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:31.456535 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 21:13:31.456661 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:31.456552 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 21:13:31.456661 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:31.456559 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dddcc6c5c-nmgkt: secret "image-registry-tls" not found Jul 06 21:13:31.456661 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:31.456619 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls podName:5bcf19dc-706f-4379-a0a1-18029d327224 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:32.456600589 +0000 UTC m=+36.581581991 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls") pod "image-registry-5dddcc6c5c-nmgkt" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224") : secret "image-registry-tls" not found Jul 06 21:13:31.456661 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:31.456637 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls podName:de552445-923f-48e7-9a47-60f066827b50 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:32.456628781 +0000 UTC m=+36.581610189 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls") pod "dns-default-w8w2r" (UID: "de552445-923f-48e7-9a47-60f066827b50") : secret "dns-default-metrics-tls" not found Jul 06 21:13:31.483907 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.483877 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:13:31.484045 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.483877 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:13:31.484099 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.483879 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:13:31.487049 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.487025 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 21:13:31.488105 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.488070 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kz87d\"" Jul 06 21:13:31.488105 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.488079 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 21:13:31.488273 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.488107 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 21:13:31.488273 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.488118 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c2dbc\"" Jul 06 21:13:31.488273 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:31.488222 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 21:13:32.089151 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:32.089102 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:32.194501 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:32.194432 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:32.475054 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:32.463873 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:32.475054 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:32.463957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:32.475054 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:32.464107 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 21:13:32.475054 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:32.464122 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dddcc6c5c-nmgkt: secret "image-registry-tls" not found Jul 06 21:13:32.475054 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:32.464182 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls podName:5bcf19dc-706f-4379-a0a1-18029d327224 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:34.4641653 +0000 UTC m=+38.589146718 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls") pod "image-registry-5dddcc6c5c-nmgkt" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224") : secret "image-registry-tls" not found Jul 06 21:13:32.475054 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:32.464648 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 21:13:32.475054 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:32.464700 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls podName:de552445-923f-48e7-9a47-60f066827b50 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:34.464684276 +0000 UTC m=+38.589665683 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls") pod "dns-default-w8w2r" (UID: "de552445-923f-48e7-9a47-60f066827b50") : secret "dns-default-metrics-tls" not found Jul 06 21:13:32.538823 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:32.538800 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dwg5d"] Jul 06 21:13:32.544245 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:13:32.544201 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78439b85_9901_45a4_898d_dbeb91d2eacc.slice/crio-3be79d8a8366067808836d5b26e053cdbe6e4b29712a5a5e51b061c8ed1ff2ec WatchSource:0}: Error finding container 3be79d8a8366067808836d5b26e053cdbe6e4b29712a5a5e51b061c8ed1ff2ec: Status 404 returned error can't find the container with id 3be79d8a8366067808836d5b26e053cdbe6e4b29712a5a5e51b061c8ed1ff2ec Jul 06 21:13:32.714184 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:32.714155 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dwg5d" event={"ID":"78439b85-9901-45a4-898d-dbeb91d2eacc","Type":"ContainerStarted","Data":"3be79d8a8366067808836d5b26e053cdbe6e4b29712a5a5e51b061c8ed1ff2ec"} Jul 06 21:13:32.716689 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:32.716662 2577 generic.go:356] "Generic (PLEG): container finished" podID="d291c60c-4d03-4c4d-8284-0f3d918d5ad1" containerID="1acc88d5557e592e00b312026e340b0284416c38ac8f5cc8d92a5c34ef5c19df" exitCode=0 Jul 06 21:13:32.716790 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:32.716711 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7ds6m" event={"ID":"d291c60c-4d03-4c4d-8284-0f3d918d5ad1","Type":"ContainerDied","Data":"1acc88d5557e592e00b312026e340b0284416c38ac8f5cc8d92a5c34ef5c19df"} Jul 06 21:13:33.091415 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.091246 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6"] Jul 06 21:13:33.095015 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.093603 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:33.096431 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.096406 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2nb5d\"" Jul 06 21:13:33.097282 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.097228 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 21:13:33.097410 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.097291 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 21:13:33.097410 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.097353 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 21:13:33.098651 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.098634 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 21:13:33.108396 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.108375 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6"] Jul 06 21:13:33.170229 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.170200 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx"] Jul 06 21:13:33.172177 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.172163 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-p4mff"] Jul 06 21:13:33.172289 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.172273 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:13:33.174652 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.174636 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.175106 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.175089 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 21:13:33.175183 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.175127 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-s242h\"" Jul 06 21:13:33.175183 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.175143 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 21:13:33.175183 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.175157 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 21:13:33.178235 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.178205 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 21:13:33.178342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.178261 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 21:13:33.178404 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.178208 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 21:13:33.178961 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.178780 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 21:13:33.179168 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.178971 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gd7zs\"" Jul 06 21:13:33.183555 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.183534 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 21:13:33.188785 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.188765 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx"] Jul 06 21:13:33.189760 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.189741 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-p4mff"] Jul 06 21:13:33.271123 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.271085 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h92sx\" (UID: \"04acc29b-4c65-4068-b94d-99f8a680b6be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:13:33.271293 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.271244 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f922a864-b75a-4e41-afa7-d868357ceff9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:33.271346 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.271293 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5b436268-66a4-40f6-9386-1d0ee9b180da-trusted-ca\") pod \"console-operator-68b757fb67-p4mff\" (UID: \"5b436268-66a4-40f6-9386-1d0ee9b180da\") " pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.271401 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.271337 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b436268-66a4-40f6-9386-1d0ee9b180da-config\") pod \"console-operator-68b757fb67-p4mff\" (UID: \"5b436268-66a4-40f6-9386-1d0ee9b180da\") " pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.271401 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.271367 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b436268-66a4-40f6-9386-1d0ee9b180da-serving-cert\") pod \"console-operator-68b757fb67-p4mff\" (UID: \"5b436268-66a4-40f6-9386-1d0ee9b180da\") " pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.271529 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.271413 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8xgg\" (UniqueName: \"kubernetes.io/projected/5b436268-66a4-40f6-9386-1d0ee9b180da-kube-api-access-q8xgg\") pod \"console-operator-68b757fb67-p4mff\" (UID: \"5b436268-66a4-40f6-9386-1d0ee9b180da\") " pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.271529 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.271459 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjg79\" (UniqueName: \"kubernetes.io/projected/04acc29b-4c65-4068-b94d-99f8a680b6be-kube-api-access-bjg79\") pod \"cluster-samples-operator-56fcf86bf-h92sx\" (UID: \"04acc29b-4c65-4068-b94d-99f8a680b6be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:13:33.271529 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.271497 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:33.271647 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.271562 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfdvk\" (UniqueName: \"kubernetes.io/projected/f922a864-b75a-4e41-afa7-d868357ceff9-kube-api-access-xfdvk\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:33.334276 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.334190 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:33.372334 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.372246 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h92sx\" (UID: \"04acc29b-4c65-4068-b94d-99f8a680b6be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:13:33.372512 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.372343 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f922a864-b75a-4e41-afa7-d868357ceff9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:33.372512 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.372374 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5b436268-66a4-40f6-9386-1d0ee9b180da-trusted-ca\") pod \"console-operator-68b757fb67-p4mff\" (UID: \"5b436268-66a4-40f6-9386-1d0ee9b180da\") " pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.372512 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.372403 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b436268-66a4-40f6-9386-1d0ee9b180da-config\") pod \"console-operator-68b757fb67-p4mff\" (UID: \"5b436268-66a4-40f6-9386-1d0ee9b180da\") " pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.372512 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:33.372413 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 21:13:33.372512 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.372426 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b436268-66a4-40f6-9386-1d0ee9b180da-serving-cert\") pod \"console-operator-68b757fb67-p4mff\" (UID: \"5b436268-66a4-40f6-9386-1d0ee9b180da\") " pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.372512 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.372447 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8xgg\" (UniqueName: \"kubernetes.io/projected/5b436268-66a4-40f6-9386-1d0ee9b180da-kube-api-access-q8xgg\") pod \"console-operator-68b757fb67-p4mff\" (UID: \"5b436268-66a4-40f6-9386-1d0ee9b180da\") " pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.372512 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.372496 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjg79\" (UniqueName: \"kubernetes.io/projected/04acc29b-4c65-4068-b94d-99f8a680b6be-kube-api-access-bjg79\") pod \"cluster-samples-operator-56fcf86bf-h92sx\" (UID: \"04acc29b-4c65-4068-b94d-99f8a680b6be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:13:33.372512 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:33.372515 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls podName:04acc29b-4c65-4068-b94d-99f8a680b6be nodeName:}" failed. No retries permitted until 2026-07-06 21:13:33.872493749 +0000 UTC m=+37.997475153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-h92sx" (UID: "04acc29b-4c65-4068-b94d-99f8a680b6be") : secret "samples-operator-tls" not found Jul 06 21:13:33.372901 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.372562 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:33.372901 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.372598 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfdvk\" (UniqueName: \"kubernetes.io/projected/f922a864-b75a-4e41-afa7-d868357ceff9-kube-api-access-xfdvk\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:33.373209 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:33.373136 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 21:13:33.373209 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.373170 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b436268-66a4-40f6-9386-1d0ee9b180da-config\") pod \"console-operator-68b757fb67-p4mff\" (UID: \"5b436268-66a4-40f6-9386-1d0ee9b180da\") " pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.373209 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:33.373186 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls podName:f922a864-b75a-4e41-afa7-d868357ceff9 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:33.873172511 +0000 UTC m=+37.998153924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c6wd6" (UID: "f922a864-b75a-4e41-afa7-d868357ceff9") : secret "cluster-monitoring-operator-tls" not found Jul 06 21:13:33.373209 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.373187 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f922a864-b75a-4e41-afa7-d868357ceff9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:33.373987 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.373948 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5b436268-66a4-40f6-9386-1d0ee9b180da-trusted-ca\") pod \"console-operator-68b757fb67-p4mff\" (UID: \"5b436268-66a4-40f6-9386-1d0ee9b180da\") " pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.377151 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.377129 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b436268-66a4-40f6-9386-1d0ee9b180da-serving-cert\") pod \"console-operator-68b757fb67-p4mff\" (UID: \"5b436268-66a4-40f6-9386-1d0ee9b180da\") " pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.390153 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.390132 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjg79\" (UniqueName: \"kubernetes.io/projected/04acc29b-4c65-4068-b94d-99f8a680b6be-kube-api-access-bjg79\") pod \"cluster-samples-operator-56fcf86bf-h92sx\" (UID: \"04acc29b-4c65-4068-b94d-99f8a680b6be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:13:33.393394 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.393368 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8xgg\" (UniqueName: \"kubernetes.io/projected/5b436268-66a4-40f6-9386-1d0ee9b180da-kube-api-access-q8xgg\") pod \"console-operator-68b757fb67-p4mff\" (UID: \"5b436268-66a4-40f6-9386-1d0ee9b180da\") " pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.393580 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.393563 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfdvk\" (UniqueName: \"kubernetes.io/projected/f922a864-b75a-4e41-afa7-d868357ceff9-kube-api-access-xfdvk\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:33.433779 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.433737 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:33.490011 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.489981 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:33.645355 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.645276 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-p4mff"] Jul 06 21:13:33.648639 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:13:33.648590 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b436268_66a4_40f6_9386_1d0ee9b180da.slice/crio-91b22bacf67081f6c57e010220c94341e01d1bccaf8c84a84d7b9d27b34cc231 WatchSource:0}: Error finding container 91b22bacf67081f6c57e010220c94341e01d1bccaf8c84a84d7b9d27b34cc231: Status 404 returned error can't find the container with id 91b22bacf67081f6c57e010220c94341e01d1bccaf8c84a84d7b9d27b34cc231 Jul 06 21:13:33.723878 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.723842 2577 generic.go:356] "Generic (PLEG): container finished" podID="d291c60c-4d03-4c4d-8284-0f3d918d5ad1" containerID="ae3b180df29996394f10eaca01e8a9a061a2116281b91bffaa3ec0a7d5e9fcab" exitCode=0 Jul 06 21:13:33.724067 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.723924 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7ds6m" event={"ID":"d291c60c-4d03-4c4d-8284-0f3d918d5ad1","Type":"ContainerDied","Data":"ae3b180df29996394f10eaca01e8a9a061a2116281b91bffaa3ec0a7d5e9fcab"} Jul 06 21:13:33.725296 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.725266 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" event={"ID":"5b436268-66a4-40f6-9386-1d0ee9b180da","Type":"ContainerStarted","Data":"91b22bacf67081f6c57e010220c94341e01d1bccaf8c84a84d7b9d27b34cc231"} Jul 06 21:13:33.878709 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.878682 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h92sx\" (UID: \"04acc29b-4c65-4068-b94d-99f8a680b6be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:13:33.878815 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:33.878786 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:33.878882 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:33.878854 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 21:13:33.878932 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:33.878924 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls podName:04acc29b-4c65-4068-b94d-99f8a680b6be nodeName:}" failed. No retries permitted until 2026-07-06 21:13:34.878903693 +0000 UTC m=+39.003885097 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-h92sx" (UID: "04acc29b-4c65-4068-b94d-99f8a680b6be") : secret "samples-operator-tls" not found Jul 06 21:13:33.878989 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:33.878923 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 21:13:33.878989 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:33.878962 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls podName:f922a864-b75a-4e41-afa7-d868357ceff9 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:34.878952792 +0000 UTC m=+39.003934193 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c6wd6" (UID: "f922a864-b75a-4e41-afa7-d868357ceff9") : secret "cluster-monitoring-operator-tls" not found Jul 06 21:13:34.484042 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:34.483996 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:34.484927 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:34.484091 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:34.484927 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:34.484162 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 21:13:34.484927 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:34.484216 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 21:13:34.484927 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:34.484229 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dddcc6c5c-nmgkt: secret "image-registry-tls" not found Jul 06 21:13:34.484927 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:34.484235 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls podName:de552445-923f-48e7-9a47-60f066827b50 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:38.484214876 +0000 UTC m=+42.609196291 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls") pod "dns-default-w8w2r" (UID: "de552445-923f-48e7-9a47-60f066827b50") : secret "dns-default-metrics-tls" not found Jul 06 21:13:34.484927 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:34.484269 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls podName:5bcf19dc-706f-4379-a0a1-18029d327224 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:38.484258465 +0000 UTC m=+42.609239866 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls") pod "image-registry-5dddcc6c5c-nmgkt" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224") : secret "image-registry-tls" not found Jul 06 21:13:34.732780 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:34.732737 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7ds6m" event={"ID":"d291c60c-4d03-4c4d-8284-0f3d918d5ad1","Type":"ContainerStarted","Data":"5c516c465a2ebea84b9bdb72e0a38c2bb54fe00f94cf5c5f7add4c2580458530"} Jul 06 21:13:34.757138 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:34.757018 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7ds6m" podStartSLOduration=8.155945795 podStartE2EDuration="38.75699791s" totalImagesPullingTime="30.601052115s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:12:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:34.755635831 +0000 UTC m=+38.880617256" watchObservedRunningTime="2026-07-06 21:13:34.75699791 +0000 UTC m=+38.881979337" Jul 06 21:13:34.880599 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:34.880557 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:34.887175 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:34.887143 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:34.887316 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:34.887244 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h92sx\" (UID: \"04acc29b-4c65-4068-b94d-99f8a680b6be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:13:34.887380 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:34.887328 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 21:13:34.887436 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:34.887393 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls podName:f922a864-b75a-4e41-afa7-d868357ceff9 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:36.887376213 +0000 UTC m=+41.012357623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c6wd6" (UID: "f922a864-b75a-4e41-afa7-d868357ceff9") : secret "cluster-monitoring-operator-tls" not found Jul 06 21:13:34.887695 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:34.887669 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 21:13:34.887821 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:34.887735 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls podName:04acc29b-4c65-4068-b94d-99f8a680b6be nodeName:}" failed. No retries permitted until 2026-07-06 21:13:36.887719015 +0000 UTC m=+41.012700421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-h92sx" (UID: "04acc29b-4c65-4068-b94d-99f8a680b6be") : secret "samples-operator-tls" not found Jul 06 21:13:34.978784 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:34.978748 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:35.041387 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:35.041297 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:36.353654 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:36.353620 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:36.738082 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:36.738038 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dwg5d" event={"ID":"78439b85-9901-45a4-898d-dbeb91d2eacc","Type":"ContainerStarted","Data":"75eabb15b6fc9a128570547b1cdc03fd0a766efbe7dde25b4dbaf64853d1ee8a"} Jul 06 21:13:36.739757 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:36.739733 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/0.log" Jul 06 21:13:36.739872 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:36.739768 2577 generic.go:356] "Generic (PLEG): container finished" podID="5b436268-66a4-40f6-9386-1d0ee9b180da" containerID="965b8641938a3f2af6c3e7c1cb903d2936744b815f1bf140d442120ec69b5e91" exitCode=255 Jul 06 21:13:36.739872 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:36.739795 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" event={"ID":"5b436268-66a4-40f6-9386-1d0ee9b180da","Type":"ContainerDied","Data":"965b8641938a3f2af6c3e7c1cb903d2936744b815f1bf140d442120ec69b5e91"} Jul 06 21:13:36.739994 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:36.739980 2577 scope.go:117] "RemoveContainer" containerID="965b8641938a3f2af6c3e7c1cb903d2936744b815f1bf140d442120ec69b5e91" Jul 06 21:13:36.759583 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:36.759541 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dwg5d" podStartSLOduration=3.130304049 podStartE2EDuration="6.759517322s" totalImagesPullingTime="3.629213273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:13:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:36.757199412 +0000 UTC m=+40.882180831" watchObservedRunningTime="2026-07-06 21:13:36.759517322 +0000 UTC m=+40.884498780" Jul 06 21:13:36.902254 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:36.902216 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h92sx\" (UID: \"04acc29b-4c65-4068-b94d-99f8a680b6be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:13:36.903041 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:36.902449 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:36.903041 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:36.902481 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 21:13:36.903041 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:36.902554 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls podName:04acc29b-4c65-4068-b94d-99f8a680b6be nodeName:}" failed. No retries permitted until 2026-07-06 21:13:40.902533763 +0000 UTC m=+45.027515167 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-h92sx" (UID: "04acc29b-4c65-4068-b94d-99f8a680b6be") : secret "samples-operator-tls" not found Jul 06 21:13:36.903041 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:36.902878 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 21:13:36.903041 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:36.902912 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls podName:f922a864-b75a-4e41-afa7-d868357ceff9 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:40.902901545 +0000 UTC m=+45.027882946 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c6wd6" (UID: "f922a864-b75a-4e41-afa7-d868357ceff9") : secret "cluster-monitoring-operator-tls" not found Jul 06 21:13:37.743980 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:37.743951 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:13:37.744408 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:37.744391 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/0.log" Jul 06 21:13:37.744491 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:37.744438 2577 generic.go:356] "Generic (PLEG): container finished" podID="5b436268-66a4-40f6-9386-1d0ee9b180da" containerID="2436bf2b305d9e60bf528495f4e67ec4936677bb75f6277a9e7c667eafc7d187" exitCode=255 Jul 06 21:13:37.744598 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:37.744574 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" event={"ID":"5b436268-66a4-40f6-9386-1d0ee9b180da","Type":"ContainerDied","Data":"2436bf2b305d9e60bf528495f4e67ec4936677bb75f6277a9e7c667eafc7d187"} Jul 06 21:13:37.744671 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:37.744656 2577 scope.go:117] "RemoveContainer" containerID="965b8641938a3f2af6c3e7c1cb903d2936744b815f1bf140d442120ec69b5e91" Jul 06 21:13:37.744768 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:37.744755 2577 scope.go:117] "RemoveContainer" containerID="2436bf2b305d9e60bf528495f4e67ec4936677bb75f6277a9e7c667eafc7d187" Jul 06 21:13:37.744935 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:37.744915 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-p4mff_openshift-console-operator(5b436268-66a4-40f6-9386-1d0ee9b180da)\"" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" podUID="5b436268-66a4-40f6-9386-1d0ee9b180da" Jul 06 21:13:38.515206 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:38.515171 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:38.515419 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:38.515234 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:38.515419 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:38.515323 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 21:13:38.515419 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:38.515399 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls podName:de552445-923f-48e7-9a47-60f066827b50 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:46.515378086 +0000 UTC m=+50.640359513 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls") pod "dns-default-w8w2r" (UID: "de552445-923f-48e7-9a47-60f066827b50") : secret "dns-default-metrics-tls" not found Jul 06 21:13:38.515419 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:38.515326 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 21:13:38.515674 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:38.515430 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dddcc6c5c-nmgkt: secret "image-registry-tls" not found Jul 06 21:13:38.515674 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:38.515489 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls podName:5bcf19dc-706f-4379-a0a1-18029d327224 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:46.515472399 +0000 UTC m=+50.640453815 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls") pod "image-registry-5dddcc6c5c-nmgkt" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224") : secret "image-registry-tls" not found Jul 06 21:13:38.748103 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:38.748076 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:13:38.748586 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:38.748457 2577 scope.go:117] "RemoveContainer" containerID="2436bf2b305d9e60bf528495f4e67ec4936677bb75f6277a9e7c667eafc7d187" Jul 06 21:13:38.748682 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:38.748661 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-p4mff_openshift-console-operator(5b436268-66a4-40f6-9386-1d0ee9b180da)\"" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" podUID="5b436268-66a4-40f6-9386-1d0ee9b180da" Jul 06 21:13:39.623495 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:39.623443 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:39.684334 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:39.684293 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:40.364251 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:40.364221 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:40.433773 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:40.433713 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:40.492434 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:40.492399 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:40.932148 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:40.932118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:40.932335 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:40.932171 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h92sx\" (UID: \"04acc29b-4c65-4068-b94d-99f8a680b6be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:13:40.932335 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:40.932284 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 21:13:40.932335 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:40.932331 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 21:13:40.932439 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:40.932362 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls podName:f922a864-b75a-4e41-afa7-d868357ceff9 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:48.932342001 +0000 UTC m=+53.057323428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c6wd6" (UID: "f922a864-b75a-4e41-afa7-d868357ceff9") : secret "cluster-monitoring-operator-tls" not found Jul 06 21:13:40.932439 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:40.932379 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls podName:04acc29b-4c65-4068-b94d-99f8a680b6be nodeName:}" failed. No retries permitted until 2026-07-06 21:13:48.932371121 +0000 UTC m=+53.057352523 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-h92sx" (UID: "04acc29b-4c65-4068-b94d-99f8a680b6be") : secret "samples-operator-tls" not found Jul 06 21:13:41.229148 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.229063 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:41.285217 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.285188 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:41.377498 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.377448 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lsz7j"] Jul 06 21:13:41.409504 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.409453 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lsz7j"] Jul 06 21:13:41.409659 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.409571 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.412024 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.412002 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 21:13:41.412141 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.412063 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 21:13:41.412141 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.412129 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 21:13:41.413073 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.413055 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 21:13:41.413269 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.413253 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-spnsv\"" Jul 06 21:13:41.429309 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.429280 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:41.437170 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.437154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htr7c\" (UniqueName: \"kubernetes.io/projected/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-api-access-htr7c\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.437263 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.437182 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.437263 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.437201 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.437263 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.437219 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1e609dd-7208-45c8-93fb-019ef2780f55-data-volume\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.437358 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.437301 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1e609dd-7208-45c8-93fb-019ef2780f55-crio-socket\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.507630 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.507540 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:41.537619 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.537587 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.537619 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.537620 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.537783 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.537643 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1e609dd-7208-45c8-93fb-019ef2780f55-data-volume\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.537783 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:41.537702 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 21:13:41.537783 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.537737 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1e609dd-7208-45c8-93fb-019ef2780f55-crio-socket\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.537783 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:41.537755 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 21:13:41.537783 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:41.537771 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm podName:f1e609dd-7208-45c8-93fb-019ef2780f55 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:42.037751998 +0000 UTC m=+46.162733418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lsz7j" (UID: "f1e609dd-7208-45c8-93fb-019ef2780f55") : configmap "kube-rbac-proxy" not found Jul 06 21:13:41.537971 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:41.537815 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls podName:f1e609dd-7208-45c8-93fb-019ef2780f55 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:42.037798688 +0000 UTC m=+46.162780107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lsz7j" (UID: "f1e609dd-7208-45c8-93fb-019ef2780f55") : secret "insights-runtime-extractor-tls" not found Jul 06 21:13:41.537971 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.537888 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htr7c\" (UniqueName: \"kubernetes.io/projected/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-api-access-htr7c\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.537971 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.537923 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1e609dd-7208-45c8-93fb-019ef2780f55-crio-socket\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.538097 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.538076 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1e609dd-7208-45c8-93fb-019ef2780f55-data-volume\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.547519 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.547497 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htr7c\" (UniqueName: \"kubernetes.io/projected/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-api-access-htr7c\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:41.602325 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.602256 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:41.827598 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.827556 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:41.850076 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.850046 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-9x8sj"] Jul 06 21:13:41.873158 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.873134 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-9x8sj"] Jul 06 21:13:41.873284 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.873231 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9x8sj" Jul 06 21:13:41.875649 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.875623 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-26nbp\"" Jul 06 21:13:41.875773 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.875635 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 06 21:13:41.875773 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.875634 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 06 21:13:41.927586 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.927552 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:41.940763 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:41.940735 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pgk8\" (UniqueName: \"kubernetes.io/projected/f40d04cd-143d-44b0-9367-416c23b8d895-kube-api-access-9pgk8\") pod \"migrator-d57d58cc9-9x8sj\" (UID: \"f40d04cd-143d-44b0-9367-416c23b8d895\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9x8sj" Jul 06 21:13:42.027738 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:42.027698 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:42.041754 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:42.041726 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pgk8\" (UniqueName: \"kubernetes.io/projected/f40d04cd-143d-44b0-9367-416c23b8d895-kube-api-access-9pgk8\") pod \"migrator-d57d58cc9-9x8sj\" (UID: \"f40d04cd-143d-44b0-9367-416c23b8d895\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9x8sj" Jul 06 21:13:42.041849 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:42.041761 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:42.041849 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:42.041796 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:42.041916 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:42.041864 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 21:13:42.041966 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:42.041927 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm podName:f1e609dd-7208-45c8-93fb-019ef2780f55 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:43.041910236 +0000 UTC m=+47.166891654 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lsz7j" (UID: "f1e609dd-7208-45c8-93fb-019ef2780f55") : configmap "kube-rbac-proxy" not found Jul 06 21:13:42.042020 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:42.041969 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 21:13:42.042070 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:42.042033 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls podName:f1e609dd-7208-45c8-93fb-019ef2780f55 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:43.042016683 +0000 UTC m=+47.166998095 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lsz7j" (UID: "f1e609dd-7208-45c8-93fb-019ef2780f55") : secret "insights-runtime-extractor-tls" not found Jul 06 21:13:42.049931 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:42.049900 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pgk8\" (UniqueName: \"kubernetes.io/projected/f40d04cd-143d-44b0-9367-416c23b8d895-kube-api-access-9pgk8\") pod \"migrator-d57d58cc9-9x8sj\" (UID: \"f40d04cd-143d-44b0-9367-416c23b8d895\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9x8sj" Jul 06 21:13:42.182177 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:42.182096 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9x8sj" Jul 06 21:13:42.213827 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:42.213798 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:42.314907 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:42.314879 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:42.322897 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:42.322866 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-9x8sj"] Jul 06 21:13:42.325837 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:13:42.325812 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf40d04cd_143d_44b0_9367_416c23b8d895.slice/crio-1d9d011497fdae58e1216bfaf73e773815e6ac04dfc9b7ed61f0722b617c740f WatchSource:0}: Error finding container 1d9d011497fdae58e1216bfaf73e773815e6ac04dfc9b7ed61f0722b617c740f: Status 404 returned error can't find the container with id 1d9d011497fdae58e1216bfaf73e773815e6ac04dfc9b7ed61f0722b617c740f Jul 06 21:13:42.759544 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:42.759505 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9x8sj" event={"ID":"f40d04cd-143d-44b0-9367-416c23b8d895","Type":"ContainerStarted","Data":"1d9d011497fdae58e1216bfaf73e773815e6ac04dfc9b7ed61f0722b617c740f"} Jul 06 21:13:42.790781 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:42.790747 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:42.886507 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:42.886454 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:43.049815 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:43.049698 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:43.049815 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:43.049743 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:43.050079 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:43.049844 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 21:13:43.050079 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:43.049924 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm podName:f1e609dd-7208-45c8-93fb-019ef2780f55 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:45.049903014 +0000 UTC m=+49.174884423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lsz7j" (UID: "f1e609dd-7208-45c8-93fb-019ef2780f55") : configmap "kube-rbac-proxy" not found Jul 06 21:13:43.050079 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:43.049943 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 21:13:43.050079 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:43.050002 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls podName:f1e609dd-7208-45c8-93fb-019ef2780f55 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:45.049988343 +0000 UTC m=+49.174969744 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lsz7j" (UID: "f1e609dd-7208-45c8-93fb-019ef2780f55") : secret "insights-runtime-extractor-tls" not found Jul 06 21:13:43.182795 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:43.182759 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r9fcz_6a1be0cb-488e-4e55-855e-06abda43369b/dns-node-resolver/0.log" Jul 06 21:13:43.490728 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:43.490690 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:43.490871 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:43.490747 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:43.491104 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:43.491088 2577 scope.go:117] "RemoveContainer" containerID="2436bf2b305d9e60bf528495f4e67ec4936677bb75f6277a9e7c667eafc7d187" Jul 06 21:13:43.491265 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:43.491247 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-p4mff_openshift-console-operator(5b436268-66a4-40f6-9386-1d0ee9b180da)\"" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" podUID="5b436268-66a4-40f6-9386-1d0ee9b180da" Jul 06 21:13:43.846822 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:43.846791 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:43.908120 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:43.908091 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:43.983770 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:43.983749 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8w488_b490d547-2ba8-44f0-a6e6-898846f59cc9/node-ca/0.log" Jul 06 21:13:44.587156 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:44.587132 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dwg5d_78439b85-9901-45a4-898d-dbeb91d2eacc/serve-healthcheck-canary/0.log" Jul 06 21:13:44.766520 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:44.766486 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9x8sj" event={"ID":"f40d04cd-143d-44b0-9367-416c23b8d895","Type":"ContainerStarted","Data":"15902386add4165fa89782ef06d8302ebee94e7994d9a3edd9a8672f8587540a"} Jul 06 21:13:44.766681 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:44.766537 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9x8sj" event={"ID":"f40d04cd-143d-44b0-9367-416c23b8d895","Type":"ContainerStarted","Data":"b0a472a5fe40970de1b4bb296c4f4e1c890f74d7b30ebf09925468cbc846f26a"} Jul 06 21:13:44.784847 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:44.784799 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9x8sj" podStartSLOduration=2.002683452 podStartE2EDuration="3.78478584s" totalImagesPullingTime="1.782102388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:13:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:44.783080719 +0000 UTC m=+48.908062142" watchObservedRunningTime="2026-07-06 21:13:44.78478584 +0000 UTC m=+48.909767262" Jul 06 21:13:44.814767 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:44.814733 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:45.066177 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:45.066090 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:45.066177 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:45.066124 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:45.066598 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:45.066212 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 21:13:45.066598 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:45.066279 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm podName:f1e609dd-7208-45c8-93fb-019ef2780f55 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:49.066261882 +0000 UTC m=+53.191243286 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lsz7j" (UID: "f1e609dd-7208-45c8-93fb-019ef2780f55") : configmap "kube-rbac-proxy" not found Jul 06 21:13:45.066598 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:45.066287 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 21:13:45.066598 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:45.066326 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls podName:f1e609dd-7208-45c8-93fb-019ef2780f55 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:49.066315899 +0000 UTC m=+53.191297304 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lsz7j" (UID: "f1e609dd-7208-45c8-93fb-019ef2780f55") : secret "insights-runtime-extractor-tls" not found Jul 06 21:13:46.578645 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:46.578609 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:13:46.579126 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:46.578695 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:13:46.579126 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:46.578792 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 21:13:46.579126 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:46.578818 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 21:13:46.579126 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:46.578829 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dddcc6c5c-nmgkt: secret "image-registry-tls" not found Jul 06 21:13:46.579126 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:46.578881 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls podName:de552445-923f-48e7-9a47-60f066827b50 nodeName:}" failed. No retries permitted until 2026-07-06 21:14:02.578856472 +0000 UTC m=+66.703837875 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls") pod "dns-default-w8w2r" (UID: "de552445-923f-48e7-9a47-60f066827b50") : secret "dns-default-metrics-tls" not found Jul 06 21:13:46.579126 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:46.578903 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls podName:5bcf19dc-706f-4379-a0a1-18029d327224 nodeName:}" failed. No retries permitted until 2026-07-06 21:14:02.578891652 +0000 UTC m=+66.703873052 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls") pod "image-registry-5dddcc6c5c-nmgkt" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224") : secret "image-registry-tls" not found Jul 06 21:13:46.626759 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:46.626715 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:46.976891 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:46.976805 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:47.077449 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:47.077418 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:48.200846 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:48.200809 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:49.000707 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:49.000665 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h92sx\" (UID: \"04acc29b-4c65-4068-b94d-99f8a680b6be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:13:49.000883 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:49.000766 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:13:49.000883 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:49.000826 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 21:13:49.001020 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:49.000897 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 21:13:49.001020 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:49.000900 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls podName:04acc29b-4c65-4068-b94d-99f8a680b6be nodeName:}" failed. No retries permitted until 2026-07-06 21:14:05.000884301 +0000 UTC m=+69.125865721 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-h92sx" (UID: "04acc29b-4c65-4068-b94d-99f8a680b6be") : secret "samples-operator-tls" not found Jul 06 21:13:49.001020 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:49.000949 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls podName:f922a864-b75a-4e41-afa7-d868357ceff9 nodeName:}" failed. No retries permitted until 2026-07-06 21:14:05.000938064 +0000 UTC m=+69.125919485 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-c6wd6" (UID: "f922a864-b75a-4e41-afa7-d868357ceff9") : secret "cluster-monitoring-operator-tls" not found Jul 06 21:13:49.101940 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:49.101904 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:49.101940 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:49.101940 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:49.102136 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:49.102046 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 21:13:49.102136 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:13:49.102100 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls podName:f1e609dd-7208-45c8-93fb-019ef2780f55 nodeName:}" failed. No retries permitted until 2026-07-06 21:13:57.10208425 +0000 UTC m=+61.227065662 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lsz7j" (UID: "f1e609dd-7208-45c8-93fb-019ef2780f55") : secret "insights-runtime-extractor-tls" not found Jul 06 21:13:49.102502 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:49.102482 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e609dd-7208-45c8-93fb-019ef2780f55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:49.117733 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:49.117670 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:49.215930 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:49.215849 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:49.318489 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:49.318383 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:49.417826 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:49.417792 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:49.484434 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:49.484401 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:50.009246 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:50.009184 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:50.615755 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:50.614805 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:51.249614 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:51.249579 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:51.418350 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:51.418315 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:51.811685 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:51.811646 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:52.015288 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:52.015257 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:52.409808 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:52.409775 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:52.468916 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:52.468883 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:53.067368 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:53.067331 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:53.210844 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:53.210792 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:53.612020 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:53.611990 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:53.813486 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:53.813435 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:53.873452 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:53.873361 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:54.026228 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:54.026187 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:54.214667 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:54.214577 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:54.628700 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:54.628671 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:54.686398 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:54.686363 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:55.012916 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:55.012823 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:55.621924 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:55.621888 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:55.816919 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:55.816887 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:56.010003 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:56.009917 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:56.120528 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:56.120494 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:56.414897 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:56.414863 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:56.485905 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:56.485874 2577 scope.go:117] "RemoveContainer" containerID="2436bf2b305d9e60bf528495f4e67ec4936677bb75f6277a9e7c667eafc7d187" Jul 06 21:13:56.798488 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:56.798394 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:13:56.798937 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:56.798494 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" event={"ID":"5b436268-66a4-40f6-9386-1d0ee9b180da","Type":"ContainerStarted","Data":"d6e5f0b1de9420d68e06670a3afe11c42dd13eab45aaeb0bfddd189bb8d1a82e"} Jul 06 21:13:56.798937 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:56.798751 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:56.815398 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:56.815347 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" podStartSLOduration=21.286599523 podStartE2EDuration="23.815328243s" totalImagesPullingTime="2.52872872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:13:56.814862851 +0000 UTC m=+60.939844274" watchObservedRunningTime="2026-07-06 21:13:56.815328243 +0000 UTC m=+60.940309668" Jul 06 21:13:57.021614 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.021586 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:57.151826 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.151797 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-p4mff" Jul 06 21:13:57.163980 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.163948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:57.166458 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.166433 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e609dd-7208-45c8-93fb-019ef2780f55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lsz7j\" (UID: \"f1e609dd-7208-45c8-93fb-019ef2780f55\") " pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:57.320617 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.320585 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-spnsv\"" Jul 06 21:13:57.328673 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.328646 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lsz7j" Jul 06 21:13:57.344772 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.344742 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:57.442049 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.442012 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:57.454552 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.454522 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lsz7j"] Jul 06 21:13:57.457396 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:13:57.457372 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e609dd_7208_45c8_93fb_019ef2780f55.slice/crio-26ceb78f59df55992d0c1b6fe95a3e88f6bf38ef87edeac1af51d7d6cd8e9ece WatchSource:0}: Error finding container 26ceb78f59df55992d0c1b6fe95a3e88f6bf38ef87edeac1af51d7d6cd8e9ece: Status 404 returned error can't find the container with id 26ceb78f59df55992d0c1b6fe95a3e88f6bf38ef87edeac1af51d7d6cd8e9ece Jul 06 21:13:57.500046 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.500014 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:57.617782 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.617750 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:57.683762 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.683678 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:57.712664 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.712637 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g64lt" Jul 06 21:13:57.805453 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.805418 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lsz7j" event={"ID":"f1e609dd-7208-45c8-93fb-019ef2780f55","Type":"ContainerStarted","Data":"26ceb78f59df55992d0c1b6fe95a3e88f6bf38ef87edeac1af51d7d6cd8e9ece"} Jul 06 21:13:57.812256 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.812223 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:57.877398 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:57.877347 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:58.016994 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:58.016170 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:58.086839 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:58.086804 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:58.219792 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:58.219750 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:58.283587 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:58.283499 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:58.813268 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:58.813231 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:59.214932 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:59.214877 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:59.283598 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:59.283545 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:59.413960 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:59.413912 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:59.473952 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:59.473850 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:59.615683 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:59.615624 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:59.860678 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:59.860627 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:13:59.917869 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:13:59.917825 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:00.214754 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:00.214673 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:01.819537 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:01.819499 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lsz7j" event={"ID":"f1e609dd-7208-45c8-93fb-019ef2780f55","Type":"ContainerStarted","Data":"7600c20887880843c6999a8361f24f6a2b8ff8ddcb0c484947d425ed772880e6"} Jul 06 21:14:01.819912 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:01.819569 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lsz7j" event={"ID":"f1e609dd-7208-45c8-93fb-019ef2780f55","Type":"ContainerStarted","Data":"0dc50daab41d8015faed508036d16d172d51849702bd3948b541bffbba2cd53b"} Jul 06 21:14:02.106362 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.106314 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:14:02.109059 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.109042 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 21:14:02.119309 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.119273 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c8194ca3-5e42-4623-aabb-091c40445eaa-original-pull-secret\") pod \"global-pull-secret-syncer-xjshg\" (UID: \"c8194ca3-5e42-4623-aabb-091c40445eaa\") " pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:14:02.206787 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.206754 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs\") pod \"network-metrics-daemon-xv8m9\" (UID: \"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150\") " pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:14:02.209276 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.209257 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 21:14:02.219281 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.219258 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54c01a9d-5eb8-4b7d-959a-f2a6e6a43150-metrics-certs\") pod \"network-metrics-daemon-xv8m9\" (UID: \"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150\") " pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:14:02.308052 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.308001 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x64m\" (UniqueName: \"kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m\") pod \"network-check-target-mzmmk\" (UID: \"b821193f-628f-4ed3-b87d-e96f3c441ff7\") " pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:14:02.310637 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.310618 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 21:14:02.321300 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.321283 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 21:14:02.331757 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.331737 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x64m\" (UniqueName: \"kubernetes.io/projected/b821193f-628f-4ed3-b87d-e96f3c441ff7-kube-api-access-7x64m\") pod \"network-check-target-mzmmk\" (UID: \"b821193f-628f-4ed3-b87d-e96f3c441ff7\") " pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:14:02.396015 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.395936 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjshg" Jul 06 21:14:02.405598 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.405579 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kz87d\"" Jul 06 21:14:02.411586 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.411547 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c2dbc\"" Jul 06 21:14:02.413652 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.413615 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:14:02.419900 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.419878 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xv8m9" Jul 06 21:14:02.544876 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.544848 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xjshg"] Jul 06 21:14:02.561667 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.561644 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mzmmk"] Jul 06 21:14:02.564134 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:14:02.564112 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb821193f_628f_4ed3_b87d_e96f3c441ff7.slice/crio-92a79ca3f4d097f01cdab1dae9301c3cefc166ceddc2946841f1ac4904991e4e WatchSource:0}: Error finding container 92a79ca3f4d097f01cdab1dae9301c3cefc166ceddc2946841f1ac4904991e4e: Status 404 returned error can't find the container with id 92a79ca3f4d097f01cdab1dae9301c3cefc166ceddc2946841f1ac4904991e4e Jul 06 21:14:02.581324 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.580870 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xv8m9"] Jul 06 21:14:02.584487 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:14:02.584448 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54c01a9d_5eb8_4b7d_959a_f2a6e6a43150.slice/crio-ed1128d06c25a54ac52090979c46d91232b5cf9f04bd74f3c2ba4d4e2afc1239 WatchSource:0}: Error finding container ed1128d06c25a54ac52090979c46d91232b5cf9f04bd74f3c2ba4d4e2afc1239: Status 404 returned error can't find the container with id ed1128d06c25a54ac52090979c46d91232b5cf9f04bd74f3c2ba4d4e2afc1239 Jul 06 21:14:02.610164 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.610137 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:14:02.610259 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.610204 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:14:02.612366 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.612342 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de552445-923f-48e7-9a47-60f066827b50-metrics-tls\") pod \"dns-default-w8w2r\" (UID: \"de552445-923f-48e7-9a47-60f066827b50\") " pod="openshift-dns/dns-default-w8w2r" Jul 06 21:14:02.612448 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.612415 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls\") pod \"image-registry-5dddcc6c5c-nmgkt\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:14:02.786152 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.786069 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dgf9v\"" Jul 06 21:14:02.793661 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.793626 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4w8fd\"" Jul 06 21:14:02.793801 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.793666 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:14:02.802271 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.802241 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w8w2r" Jul 06 21:14:02.824339 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.824302 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mzmmk" event={"ID":"b821193f-628f-4ed3-b87d-e96f3c441ff7","Type":"ContainerStarted","Data":"92a79ca3f4d097f01cdab1dae9301c3cefc166ceddc2946841f1ac4904991e4e"} Jul 06 21:14:02.826004 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.825872 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xv8m9" event={"ID":"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150","Type":"ContainerStarted","Data":"ed1128d06c25a54ac52090979c46d91232b5cf9f04bd74f3c2ba4d4e2afc1239"} Jul 06 21:14:02.827342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.827315 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xjshg" event={"ID":"c8194ca3-5e42-4623-aabb-091c40445eaa","Type":"ContainerStarted","Data":"cf0a8d7f4db5de16f21d18be6ccbc610c18f619fe60a22a6c3298d9ceb023075"} Jul 06 21:14:02.962575 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.962227 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dddcc6c5c-nmgkt"] Jul 06 21:14:02.967426 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:14:02.967035 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bcf19dc_706f_4379_a0a1_18029d327224.slice/crio-68049bc9cf788eb022f9874ccd1302b672f94207110519a7ae54319e46413c43 WatchSource:0}: Error finding container 68049bc9cf788eb022f9874ccd1302b672f94207110519a7ae54319e46413c43: Status 404 returned error can't find the container with id 68049bc9cf788eb022f9874ccd1302b672f94207110519a7ae54319e46413c43 Jul 06 21:14:02.981639 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:02.981611 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w8w2r"] Jul 06 21:14:02.985057 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:14:02.985031 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde552445_923f_48e7_9a47_60f066827b50.slice/crio-0e2b2cccac5baf50583d1e14abb3571e2c697be43e3ce888adf1e9feadd83e67 WatchSource:0}: Error finding container 0e2b2cccac5baf50583d1e14abb3571e2c697be43e3ce888adf1e9feadd83e67: Status 404 returned error can't find the container with id 0e2b2cccac5baf50583d1e14abb3571e2c697be43e3ce888adf1e9feadd83e67 Jul 06 21:14:03.833081 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:03.832975 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w8w2r" event={"ID":"de552445-923f-48e7-9a47-60f066827b50","Type":"ContainerStarted","Data":"0e2b2cccac5baf50583d1e14abb3571e2c697be43e3ce888adf1e9feadd83e67"} Jul 06 21:14:03.837141 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:03.836268 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" event={"ID":"5bcf19dc-706f-4379-a0a1-18029d327224","Type":"ContainerStarted","Data":"cf0dd3d8c371e3c2451cad99c57569ad93aaef12770186b48da98ce6049dce6c"} Jul 06 21:14:03.837141 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:03.836312 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" event={"ID":"5bcf19dc-706f-4379-a0a1-18029d327224","Type":"ContainerStarted","Data":"68049bc9cf788eb022f9874ccd1302b672f94207110519a7ae54319e46413c43"} Jul 06 21:14:03.837141 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:03.837067 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:14:03.859582 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:03.859068 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" podStartSLOduration=67.859047633 podStartE2EDuration="1m7.859047633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:12:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:14:03.858523547 +0000 UTC m=+67.983504969" watchObservedRunningTime="2026-07-06 21:14:03.859047633 +0000 UTC m=+67.984029059" Jul 06 21:14:05.033990 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:05.033511 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h92sx\" (UID: \"04acc29b-4c65-4068-b94d-99f8a680b6be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:14:05.033990 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:05.033588 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:14:05.036835 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:05.036774 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04acc29b-4c65-4068-b94d-99f8a680b6be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h92sx\" (UID: \"04acc29b-4c65-4068-b94d-99f8a680b6be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:14:05.036963 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:05.036882 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f922a864-b75a-4e41-afa7-d868357ceff9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-c6wd6\" (UID: \"f922a864-b75a-4e41-afa7-d868357ceff9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:14:05.206340 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:05.206292 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2nb5d\"" Jul 06 21:14:05.214039 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:05.213912 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" Jul 06 21:14:05.285787 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:05.285699 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-s242h\"" Jul 06 21:14:05.293304 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:05.293276 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" Jul 06 21:14:08.656674 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.655393 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:08.746704 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.746625 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6"] Jul 06 21:14:08.759569 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.759543 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx"] Jul 06 21:14:08.808626 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.808584 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:08.856378 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.856338 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lsz7j" event={"ID":"f1e609dd-7208-45c8-93fb-019ef2780f55","Type":"ContainerStarted","Data":"73c9253083309becc8b12e031c09350192d100eb0fb406bff85df7f03f643113"} Jul 06 21:14:08.858181 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.858139 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mzmmk" event={"ID":"b821193f-628f-4ed3-b87d-e96f3c441ff7","Type":"ContainerStarted","Data":"0a1a9869609c584e786824a3c8075a5e976268a6197ed00f31c8751cc2275ce8"} Jul 06 21:14:08.858373 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.858350 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:14:08.860250 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.860223 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w8w2r" event={"ID":"de552445-923f-48e7-9a47-60f066827b50","Type":"ContainerStarted","Data":"eca276673ddd9a879505ae39cb7220ebd9ee4814cae374c9e46b6cea5d36acf5"} Jul 06 21:14:08.860369 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.860271 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w8w2r" event={"ID":"de552445-923f-48e7-9a47-60f066827b50","Type":"ContainerStarted","Data":"6cdeaa1d679fb6238e03509d7255d9f2aa90ea1079ff4aa6fdcf02cc511ee9e0"} Jul 06 21:14:08.860369 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.860313 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-w8w2r" Jul 06 21:14:08.862856 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.862447 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xv8m9" event={"ID":"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150","Type":"ContainerStarted","Data":"51135064b387f2a1ab43309e02d5c1b0c96ec8c88885fbdac334ee60d843bcad"} Jul 06 21:14:08.862856 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.862500 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xv8m9" event={"ID":"54c01a9d-5eb8-4b7d-959a-f2a6e6a43150","Type":"ContainerStarted","Data":"424104c41b2ffc515c13ded794cee8b29ca88a51559d5bb5bde0e82f13f880ca"} Jul 06 21:14:08.864251 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.864185 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" event={"ID":"04acc29b-4c65-4068-b94d-99f8a680b6be","Type":"ContainerStarted","Data":"24563b2f3f71d7248e7185d5993fc0b40a58bf3d26fb588e16e383d719330822"} Jul 06 21:14:08.866120 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.866096 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xjshg" event={"ID":"c8194ca3-5e42-4623-aabb-091c40445eaa","Type":"ContainerStarted","Data":"6dc17a66e201922e0f20ad9c3d0fa347ebfbfc4d3ee5d3a2c91fc30e0e8d5229"} Jul 06 21:14:08.867337 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.867320 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" event={"ID":"f922a864-b75a-4e41-afa7-d868357ceff9","Type":"ContainerStarted","Data":"c2926b31d75b6a9330dd2f152c61328653a19d01a27d7e939f51f905bcea5d08"} Jul 06 21:14:08.881145 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.881099 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lsz7j" podStartSLOduration=16.928671576 podStartE2EDuration="27.881085484s" totalImagesPullingTime="10.952413908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:13:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:14:08.88026619 +0000 UTC m=+73.005247615" watchObservedRunningTime="2026-07-06 21:14:08.881085484 +0000 UTC m=+73.006066907" Jul 06 21:14:08.900817 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.900763 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mzmmk" podStartSLOduration=66.942421625 podStartE2EDuration="1m12.900749003s" totalImagesPullingTime="5.958327378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:14:08.89978956 +0000 UTC m=+73.024770983" watchObservedRunningTime="2026-07-06 21:14:08.900749003 +0000 UTC m=+73.025730416" Jul 06 21:14:08.906722 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.906683 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:08.928024 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.927964 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-w8w2r" podStartSLOduration=33.4003099 podStartE2EDuration="38.927947739s" totalImagesPullingTime="5.527637839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:13:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:14:08.92682443 +0000 UTC m=+73.051805845" watchObservedRunningTime="2026-07-06 21:14:08.927947739 +0000 UTC m=+73.052929231" Jul 06 21:14:08.950983 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.950919 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xv8m9" podStartSLOduration=67.036249701 podStartE2EDuration="1m12.950897069s" totalImagesPullingTime="5.914647368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:14:08.950140434 +0000 UTC m=+73.075121855" watchObservedRunningTime="2026-07-06 21:14:08.950897069 +0000 UTC m=+73.075878494" Jul 06 21:14:08.969736 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:08.969681 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xjshg" podStartSLOduration=66.995272431 podStartE2EDuration="1m12.969665314s" totalImagesPullingTime="5.974392883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:14:08.969047914 +0000 UTC m=+73.094029337" watchObservedRunningTime="2026-07-06 21:14:08.969665314 +0000 UTC m=+73.094646714" Jul 06 21:14:09.008380 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:09.008290 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:09.599724 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:09.599686 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:09.698236 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:09.698192 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:09.957295 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:09.957211 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:10.027008 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:10.026969 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:11.254515 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.254484 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:11.393414 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.393384 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:11.499142 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.499111 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:11.605094 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.605064 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj"] Jul 06 21:14:11.607120 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.607104 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj" Jul 06 21:14:11.610558 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.610536 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 21:14:11.612804 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.612779 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4xkxx\"" Jul 06 21:14:11.624098 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.624069 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:11.626358 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.626337 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj"] Jul 06 21:14:11.689284 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.689199 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0762a8e1-6ebc-48c2-bbf9-07cdd20d480a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-2ntkj\" (UID: \"0762a8e1-6ebc-48c2-bbf9-07cdd20d480a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj" Jul 06 21:14:11.694550 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.694520 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:11.790628 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.790584 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0762a8e1-6ebc-48c2-bbf9-07cdd20d480a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-2ntkj\" (UID: \"0762a8e1-6ebc-48c2-bbf9-07cdd20d480a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj" Jul 06 21:14:11.790798 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:14:11.790716 2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 06 21:14:11.790798 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:14:11.790774 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0762a8e1-6ebc-48c2-bbf9-07cdd20d480a-tls-certificates podName:0762a8e1-6ebc-48c2-bbf9-07cdd20d480a nodeName:}" failed. No retries permitted until 2026-07-06 21:14:12.29075828 +0000 UTC m=+76.415739681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/0762a8e1-6ebc-48c2-bbf9-07cdd20d480a-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-2ntkj" (UID: "0762a8e1-6ebc-48c2-bbf9-07cdd20d480a") : secret "prometheus-operator-admission-webhook-tls" not found Jul 06 21:14:11.881142 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.881105 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" event={"ID":"f922a864-b75a-4e41-afa7-d868357ceff9","Type":"ContainerStarted","Data":"fa38d17991f917d99a92f08b92fc16797079ff83f6415f06d305fbc2f8ace8e4"} Jul 06 21:14:11.882754 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.882727 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" event={"ID":"04acc29b-4c65-4068-b94d-99f8a680b6be","Type":"ContainerStarted","Data":"50a0eba1bb100d9d079ad615de05433418c5faa4a550e27347b9802e6fd2cdba"} Jul 06 21:14:11.882870 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.882772 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" event={"ID":"04acc29b-4c65-4068-b94d-99f8a680b6be","Type":"ContainerStarted","Data":"bc685a7764a0d6c754d2c3dbc8e623219b05de034928e848a00f2039776e2749"} Jul 06 21:14:11.898098 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.898052 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-c6wd6" podStartSLOduration=36.599077899 podStartE2EDuration="38.898037343s" totalImagesPullingTime="2.298959444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:14:11.897013915 +0000 UTC m=+76.021995338" watchObservedRunningTime="2026-07-06 21:14:11.898037343 +0000 UTC m=+76.023018765" Jul 06 21:14:11.914786 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:11.914745 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h92sx" podStartSLOduration=36.679638355 podStartE2EDuration="38.914730009s" totalImagesPullingTime="2.235091654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:14:11.913821466 +0000 UTC m=+76.038802892" watchObservedRunningTime="2026-07-06 21:14:11.914730009 +0000 UTC m=+76.039711432" Jul 06 21:14:12.295180 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:12.295150 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0762a8e1-6ebc-48c2-bbf9-07cdd20d480a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-2ntkj\" (UID: \"0762a8e1-6ebc-48c2-bbf9-07cdd20d480a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj" Jul 06 21:14:12.297559 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:12.297537 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0762a8e1-6ebc-48c2-bbf9-07cdd20d480a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-2ntkj\" (UID: \"0762a8e1-6ebc-48c2-bbf9-07cdd20d480a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj" Jul 06 21:14:12.356474 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:12.356440 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:12.515401 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:12.515370 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj" Jul 06 21:14:12.637379 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:12.637348 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj"] Jul 06 21:14:12.640743 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:14:12.640714 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0762a8e1_6ebc_48c2_bbf9_07cdd20d480a.slice/crio-adb8fdde07f0fae5347040445953d8ae673469195eb6b4ee0d7612ec9c71e99f WatchSource:0}: Error finding container adb8fdde07f0fae5347040445953d8ae673469195eb6b4ee0d7612ec9c71e99f: Status 404 returned error can't find the container with id adb8fdde07f0fae5347040445953d8ae673469195eb6b4ee0d7612ec9c71e99f Jul 06 21:14:12.887267 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:12.887230 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj" event={"ID":"0762a8e1-6ebc-48c2-bbf9-07cdd20d480a","Type":"ContainerStarted","Data":"adb8fdde07f0fae5347040445953d8ae673469195eb6b4ee0d7612ec9c71e99f"} Jul 06 21:14:13.892352 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:13.892309 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj" event={"ID":"0762a8e1-6ebc-48c2-bbf9-07cdd20d480a","Type":"ContainerStarted","Data":"583c79d986c5151ee9a695dbe162dc3c26787d0058ad9b640fe5f91f7da76e52"} Jul 06 21:14:13.892760 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:13.892521 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj" Jul 06 21:14:13.897035 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:13.897013 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj" Jul 06 21:14:13.928521 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:13.928456 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-2ntkj" podStartSLOduration=1.874558454 podStartE2EDuration="2.928441795s" totalImagesPullingTime="1.053883341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:14:13.909642287 +0000 UTC m=+78.034623710" watchObservedRunningTime="2026-07-06 21:14:13.928441795 +0000 UTC m=+78.053423219" Jul 06 21:14:14.595356 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:14.595322 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:14.672998 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:14.672954 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:15.407542 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:15.407509 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:15.639225 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:15.639185 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:17.274624 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:17.274590 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:17.371289 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:17.371254 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:17.471387 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:17.471349 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:17.850258 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:17.850218 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:18.874841 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:18.874807 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-w8w2r" Jul 06 21:14:21.086649 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:21.086620 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:22.764981 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:22.764942 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:22.994832 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:22.994794 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:23.032416 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.032346 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp"] Jul 06 21:14:23.036705 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.036686 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.043270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.041494 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 21:14:23.043270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.042073 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hwz75\"" Jul 06 21:14:23.043270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.043092 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dxwml"] Jul 06 21:14:23.043270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.043199 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 21:14:23.043671 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.043296 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 21:14:23.047076 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.047058 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.049420 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.049392 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp"] Jul 06 21:14:23.049548 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.049530 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 21:14:23.049750 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.049730 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 21:14:23.049846 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.049763 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 21:14:23.050026 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.050008 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6nbt4\"" Jul 06 21:14:23.070217 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.070195 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch"] Jul 06 21:14:23.073190 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.073175 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.076092 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.076043 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 21:14:23.076187 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.076149 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 21:14:23.076187 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.076149 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 21:14:23.076187 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.076174 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-zqbx7\"" Jul 06 21:14:23.094217 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.094198 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch"] Jul 06 21:14:23.099799 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.099771 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:23.174159 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174125 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-accelerators-collector-config\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.174313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174167 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.174313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174195 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-tls\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.174313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174237 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0775089d-2c9e-40ae-81c1-503952781d10-metrics-client-ca\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.174414 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174308 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2133331e-43f5-4a7b-a292-820b9469769c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-tpzgp\" (UID: \"2133331e-43f5-4a7b-a292-820b9469769c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.174414 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174332 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2133331e-43f5-4a7b-a292-820b9469769c-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-tpzgp\" (UID: \"2133331e-43f5-4a7b-a292-820b9469769c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.174414 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174362 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbjcr\" (UniqueName: \"kubernetes.io/projected/0775089d-2c9e-40ae-81c1-503952781d10-kube-api-access-hbjcr\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.174414 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174387 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-wtmp\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.174414 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174403 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a25e5d09-1c8a-42b2-9cc5-edbf059149df-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.174618 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174428 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.174618 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174456 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2133331e-43f5-4a7b-a292-820b9469769c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-tpzgp\" (UID: \"2133331e-43f5-4a7b-a292-820b9469769c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.174618 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174534 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll257\" (UniqueName: \"kubernetes.io/projected/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-api-access-ll257\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.174618 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174550 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26kqh\" (UniqueName: \"kubernetes.io/projected/2133331e-43f5-4a7b-a292-820b9469769c-kube-api-access-26kqh\") pod \"openshift-state-metrics-54cf96dfcf-tpzgp\" (UID: \"2133331e-43f5-4a7b-a292-820b9469769c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.174618 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174567 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0775089d-2c9e-40ae-81c1-503952781d10-sys\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.174618 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174584 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a25e5d09-1c8a-42b2-9cc5-edbf059149df-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.174618 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174603 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0775089d-2c9e-40ae-81c1-503952781d10-root\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.174841 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174621 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-textfile\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.174841 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174669 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.174841 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.174708 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.226965 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.226933 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:23.275413 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275377 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll257\" (UniqueName: \"kubernetes.io/projected/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-api-access-ll257\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.275413 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275412 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26kqh\" (UniqueName: \"kubernetes.io/projected/2133331e-43f5-4a7b-a292-820b9469769c-kube-api-access-26kqh\") pod \"openshift-state-metrics-54cf96dfcf-tpzgp\" (UID: \"2133331e-43f5-4a7b-a292-820b9469769c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.275632 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275429 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0775089d-2c9e-40ae-81c1-503952781d10-sys\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.275632 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275456 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a25e5d09-1c8a-42b2-9cc5-edbf059149df-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.275632 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275501 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0775089d-2c9e-40ae-81c1-503952781d10-root\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.275632 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275525 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-textfile\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.275632 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275549 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.275632 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275577 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.275632 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275525 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0775089d-2c9e-40ae-81c1-503952781d10-sys\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.275967 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275646 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-accelerators-collector-config\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.275967 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:14:23.275686 2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 21:14:23.275967 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275704 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.275967 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275731 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0775089d-2c9e-40ae-81c1-503952781d10-root\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.275967 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275736 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-tls\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.275967 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:14:23.275764 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-tls podName:a25e5d09-1c8a-42b2-9cc5-edbf059149df nodeName:}" failed. No retries permitted until 2026-07-06 21:14:23.775742405 +0000 UTC m=+87.900723817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-gzzch" (UID: "a25e5d09-1c8a-42b2-9cc5-edbf059149df") : secret "kube-state-metrics-tls" not found Jul 06 21:14:23.275967 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275821 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0775089d-2c9e-40ae-81c1-503952781d10-metrics-client-ca\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.275967 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275868 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2133331e-43f5-4a7b-a292-820b9469769c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-tpzgp\" (UID: \"2133331e-43f5-4a7b-a292-820b9469769c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.275967 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275904 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2133331e-43f5-4a7b-a292-820b9469769c-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-tpzgp\" (UID: \"2133331e-43f5-4a7b-a292-820b9469769c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.275967 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275929 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbjcr\" (UniqueName: \"kubernetes.io/projected/0775089d-2c9e-40ae-81c1-503952781d10-kube-api-access-hbjcr\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.275967 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275944 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-textfile\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.275967 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275970 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-wtmp\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.276568 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:14:23.275989 2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 06 21:14:23.276568 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.275998 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a25e5d09-1c8a-42b2-9cc5-edbf059149df-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.276568 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:14:23.276033 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2133331e-43f5-4a7b-a292-820b9469769c-openshift-state-metrics-tls podName:2133331e-43f5-4a7b-a292-820b9469769c nodeName:}" failed. No retries permitted until 2026-07-06 21:14:23.776019298 +0000 UTC m=+87.901000701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2133331e-43f5-4a7b-a292-820b9469769c-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-tpzgp" (UID: "2133331e-43f5-4a7b-a292-820b9469769c") : secret "openshift-state-metrics-tls" not found Jul 06 21:14:23.276568 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.276265 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-accelerators-collector-config\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.276568 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.276288 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a25e5d09-1c8a-42b2-9cc5-edbf059149df-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.276568 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.276330 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a25e5d09-1c8a-42b2-9cc5-edbf059149df-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.276568 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.276375 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.276568 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.276411 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2133331e-43f5-4a7b-a292-820b9469769c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-tpzgp\" (UID: \"2133331e-43f5-4a7b-a292-820b9469769c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.276568 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.276421 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-wtmp\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.276568 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.276491 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.276568 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.276536 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0775089d-2c9e-40ae-81c1-503952781d10-metrics-client-ca\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.277034 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.276896 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2133331e-43f5-4a7b-a292-820b9469769c-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-tpzgp\" (UID: \"2133331e-43f5-4a7b-a292-820b9469769c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.278409 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.278386 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-tls\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.278409 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.278401 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.278592 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.278501 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0775089d-2c9e-40ae-81c1-503952781d10-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.278783 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.278762 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2133331e-43f5-4a7b-a292-820b9469769c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-tpzgp\" (UID: \"2133331e-43f5-4a7b-a292-820b9469769c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.286867 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.286799 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll257\" (UniqueName: \"kubernetes.io/projected/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-api-access-ll257\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.286867 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.286835 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbjcr\" (UniqueName: \"kubernetes.io/projected/0775089d-2c9e-40ae-81c1-503952781d10-kube-api-access-hbjcr\") pod \"node-exporter-dxwml\" (UID: \"0775089d-2c9e-40ae-81c1-503952781d10\") " pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.287025 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.287009 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26kqh\" (UniqueName: \"kubernetes.io/projected/2133331e-43f5-4a7b-a292-820b9469769c-kube-api-access-26kqh\") pod \"openshift-state-metrics-54cf96dfcf-tpzgp\" (UID: \"2133331e-43f5-4a7b-a292-820b9469769c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.325436 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.325409 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:23.357917 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.357895 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dxwml" Jul 06 21:14:23.368984 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:14:23.368961 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0775089d_2c9e_40ae_81c1_503952781d10.slice/crio-e518d617c26b4d864da28dd681390543f6ff87b37744467bdbc15edba4ccabb3 WatchSource:0}: Error finding container e518d617c26b4d864da28dd681390543f6ff87b37744467bdbc15edba4ccabb3: Status 404 returned error can't find the container with id e518d617c26b4d864da28dd681390543f6ff87b37744467bdbc15edba4ccabb3 Jul 06 21:14:23.600335 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.600298 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:23.736748 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.736720 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:23.780862 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.780835 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:23.781237 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.780882 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2133331e-43f5-4a7b-a292-820b9469769c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-tpzgp\" (UID: \"2133331e-43f5-4a7b-a292-820b9469769c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.781237 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:14:23.780980 2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 21:14:23.781237 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:14:23.781051 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-tls podName:a25e5d09-1c8a-42b2-9cc5-edbf059149df nodeName:}" failed. No retries permitted until 2026-07-06 21:14:24.781036545 +0000 UTC m=+88.906017952 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-gzzch" (UID: "a25e5d09-1c8a-42b2-9cc5-edbf059149df") : secret "kube-state-metrics-tls" not found Jul 06 21:14:23.783119 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.783100 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2133331e-43f5-4a7b-a292-820b9469769c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-tpzgp\" (UID: \"2133331e-43f5-4a7b-a292-820b9469769c\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:23.931789 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.931696 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dxwml" event={"ID":"0775089d-2c9e-40ae-81c1-503952781d10","Type":"ContainerStarted","Data":"e518d617c26b4d864da28dd681390543f6ff87b37744467bdbc15edba4ccabb3"} Jul 06 21:14:23.950795 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:23.950760 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" Jul 06 21:14:24.034332 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.034233 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:24.105208 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.105164 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp"] Jul 06 21:14:24.106303 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.106277 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:24.183094 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.183010 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:24.274610 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:14:24.274543 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2133331e_43f5_4a7b_a292_820b9469769c.slice/crio-41abb18beffc79ec6e9376cc37ec0af643d95a55cfae2774daa5eb11968dff3a WatchSource:0}: Error finding container 41abb18beffc79ec6e9376cc37ec0af643d95a55cfae2774daa5eb11968dff3a: Status 404 returned error can't find the container with id 41abb18beffc79ec6e9376cc37ec0af643d95a55cfae2774daa5eb11968dff3a Jul 06 21:14:24.618134 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.618093 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:24.673994 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.673951 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:24.789403 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.789309 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:24.791785 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.791760 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a25e5d09-1c8a-42b2-9cc5-edbf059149df-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-gzzch\" (UID: \"a25e5d09-1c8a-42b2-9cc5-edbf059149df\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:24.881437 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.881406 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" Jul 06 21:14:24.939319 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.939287 2577 generic.go:356] "Generic (PLEG): container finished" podID="0775089d-2c9e-40ae-81c1-503952781d10" containerID="80cac9203592ce476bcab6f5faef4411e934f8f44c8f114968c618ec8578172f" exitCode=0 Jul 06 21:14:24.939457 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.939357 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dxwml" event={"ID":"0775089d-2c9e-40ae-81c1-503952781d10","Type":"ContainerDied","Data":"80cac9203592ce476bcab6f5faef4411e934f8f44c8f114968c618ec8578172f"} Jul 06 21:14:24.941626 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.941601 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" event={"ID":"2133331e-43f5-4a7b-a292-820b9469769c","Type":"ContainerStarted","Data":"3e08e35f475f11ecb92b03cd0b5c6f4a0ca084ff80a99dc4c84a21885f50e779"} Jul 06 21:14:24.941754 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.941653 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" event={"ID":"2133331e-43f5-4a7b-a292-820b9469769c","Type":"ContainerStarted","Data":"d9987a7f0bcb0b5ba9a855ad5a9fd2cb3518986a9d647ba1c104a49d0f55b4ce"} Jul 06 21:14:24.941754 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:24.941674 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" event={"ID":"2133331e-43f5-4a7b-a292-820b9469769c","Type":"ContainerStarted","Data":"41abb18beffc79ec6e9376cc37ec0af643d95a55cfae2774daa5eb11968dff3a"} Jul 06 21:14:25.015169 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:25.015148 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch"] Jul 06 21:14:25.018416 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:14:25.018393 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25e5d09_1c8a_42b2_9cc5_edbf059149df.slice/crio-54d94442742c509af7b0ce9a73c49d3e5c2a823a52e4c375dcabb4c7d470fbe2 WatchSource:0}: Error finding container 54d94442742c509af7b0ce9a73c49d3e5c2a823a52e4c375dcabb4c7d470fbe2: Status 404 returned error can't find the container with id 54d94442742c509af7b0ce9a73c49d3e5c2a823a52e4c375dcabb4c7d470fbe2 Jul 06 21:14:25.032108 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:25.032080 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:25.849762 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:25.849729 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:14:25.949099 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:25.949057 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dxwml" event={"ID":"0775089d-2c9e-40ae-81c1-503952781d10","Type":"ContainerStarted","Data":"0593e0403ed4dc5c370c1ec0cb1e60f867287853aa23f07077f71b0bcfc1bb2d"} Jul 06 21:14:25.949270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:25.949122 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dxwml" event={"ID":"0775089d-2c9e-40ae-81c1-503952781d10","Type":"ContainerStarted","Data":"cb760e95066c8b5da6f5d8fec2a88e8196f605c94808e11d29a26a0f2f07e1c3"} Jul 06 21:14:25.951854 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:25.951817 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" event={"ID":"2133331e-43f5-4a7b-a292-820b9469769c","Type":"ContainerStarted","Data":"31be7a6d4227a2db97662605f1d61c0502d3b71a04b4be9565038470bb402e91"} Jul 06 21:14:25.953360 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:25.953331 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" event={"ID":"a25e5d09-1c8a-42b2-9cc5-edbf059149df","Type":"ContainerStarted","Data":"54d94442742c509af7b0ce9a73c49d3e5c2a823a52e4c375dcabb4c7d470fbe2"} Jul 06 21:14:25.968585 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:25.968534 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dxwml" podStartSLOduration=1.9477496140000001 podStartE2EDuration="2.968517271s" totalImagesPullingTime="956.198471ms" totalInitContainerRuntime="64.569186ms" isStatefulPod=true podCreationTimestamp="2026-07-06 21:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:14:25.967134891 +0000 UTC m=+90.092116325" watchObservedRunningTime="2026-07-06 21:14:25.968517271 +0000 UTC m=+90.093498695" Jul 06 21:14:25.969588 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:25.969503 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:25.984833 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:25.984786 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-tpzgp" podStartSLOduration=1.8594771429999999 podStartE2EDuration="2.984771674s" totalImagesPullingTime="1.125294531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:14:25.983770952 +0000 UTC m=+90.108752378" watchObservedRunningTime="2026-07-06 21:14:25.984771674 +0000 UTC m=+90.109753097" Jul 06 21:14:26.073025 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:26.072988 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:26.167620 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:26.167532 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:26.266895 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:26.266855 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:26.366601 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:26.366569 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:26.542801 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:26.542769 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:26.959195 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:26.959159 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" event={"ID":"a25e5d09-1c8a-42b2-9cc5-edbf059149df","Type":"ContainerStarted","Data":"bd8497b22203bde6ffd9db6a6148fc39c90e25190a5cfb1c3665fb2ec6cfdcdd"} Jul 06 21:14:26.959586 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:26.959222 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" event={"ID":"a25e5d09-1c8a-42b2-9cc5-edbf059149df","Type":"ContainerStarted","Data":"477ef0e1f2c35bdc7ee5ddf00d2dc78430954a684558828be5a4d683748e8a88"} Jul 06 21:14:26.959586 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:26.959244 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" event={"ID":"a25e5d09-1c8a-42b2-9cc5-edbf059149df","Type":"ContainerStarted","Data":"d0391bd8f6e75a665bbb1006002b901523bf328bb5197970c7e4c502152c73ed"} Jul 06 21:14:26.978133 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:26.978085 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-gzzch" podStartSLOduration=2.601484921 podStartE2EDuration="3.978070315s" totalImagesPullingTime="1.376585394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:14:26.978053128 +0000 UTC m=+91.103034550" watchObservedRunningTime="2026-07-06 21:14:26.978070315 +0000 UTC m=+91.103051738" Jul 06 21:14:27.448577 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:27.448539 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:27.521658 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:27.521623 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:27.813179 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:27.813088 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:27.890148 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:27.890112 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:28.657053 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:28.657011 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:28.757515 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:28.757454 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:29.368512 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:29.368459 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:30.269546 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:30.269510 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:30.337303 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:30.337266 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:30.666633 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:30.666587 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:30.767093 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:30.767056 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:31.178131 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:31.178096 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5dddcc6c5c-nmgkt"] Jul 06 21:14:31.257947 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:31.257907 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:31.321386 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:31.321350 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:32.227952 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:32.227907 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:32.287858 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:32.287814 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:34.484388 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:34.484355 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:34.584200 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:34.584161 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:38.134704 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:38.134661 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:38.238293 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:38.238260 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:39.615915 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:39.615871 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:39.716651 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:39.716619 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:39.875841 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:39.875756 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mzmmk" Jul 06 21:14:41.553528 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:41.553449 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:41.652964 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:41.652924 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:41.752967 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:41.752933 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:43.550346 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:43.550306 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:43.650378 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:43.650341 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:47.925096 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:47.925063 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:48.025870 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:48.025834 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:48.085392 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:48.085354 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:54.616020 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:54.615989 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:54.674084 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:54.674044 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:56.198245 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.198203 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" podUID="5bcf19dc-706f-4379-a0a1-18029d327224" containerName="registry" containerID="cri-o://cf0dd3d8c371e3c2451cad99c57569ad93aaef12770186b48da98ce6049dce6c" gracePeriod=30 Jul 06 21:14:56.434713 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.434684 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:14:56.526986 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.526909 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5bcf19dc-706f-4379-a0a1-18029d327224-registry-certificates\") pod \"5bcf19dc-706f-4379-a0a1-18029d327224\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " Jul 06 21:14:56.526986 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.526951 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnb74\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-kube-api-access-hnb74\") pod \"5bcf19dc-706f-4379-a0a1-18029d327224\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " Jul 06 21:14:56.526986 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.526979 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5bcf19dc-706f-4379-a0a1-18029d327224-ca-trust-extracted\") pod \"5bcf19dc-706f-4379-a0a1-18029d327224\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " Jul 06 21:14:56.527247 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.527008 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5bcf19dc-706f-4379-a0a1-18029d327224-trusted-ca\") pod \"5bcf19dc-706f-4379-a0a1-18029d327224\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " Jul 06 21:14:56.527247 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.527033 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-bound-sa-token\") pod \"5bcf19dc-706f-4379-a0a1-18029d327224\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " Jul 06 21:14:56.527247 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.527079 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5bcf19dc-706f-4379-a0a1-18029d327224-image-registry-private-configuration\") pod \"5bcf19dc-706f-4379-a0a1-18029d327224\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " Jul 06 21:14:56.527247 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.527124 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls\") pod \"5bcf19dc-706f-4379-a0a1-18029d327224\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " Jul 06 21:14:56.527247 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.527163 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5bcf19dc-706f-4379-a0a1-18029d327224-installation-pull-secrets\") pod \"5bcf19dc-706f-4379-a0a1-18029d327224\" (UID: \"5bcf19dc-706f-4379-a0a1-18029d327224\") " Jul 06 21:14:56.527599 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.527497 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bcf19dc-706f-4379-a0a1-18029d327224-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5bcf19dc-706f-4379-a0a1-18029d327224" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:14:56.527814 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.527772 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bcf19dc-706f-4379-a0a1-18029d327224-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5bcf19dc-706f-4379-a0a1-18029d327224" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:14:56.529873 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.529820 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bcf19dc-706f-4379-a0a1-18029d327224-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5bcf19dc-706f-4379-a0a1-18029d327224" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:14:56.529982 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.529915 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-kube-api-access-hnb74" (OuterVolumeSpecName: "kube-api-access-hnb74") pod "5bcf19dc-706f-4379-a0a1-18029d327224" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224"). InnerVolumeSpecName "kube-api-access-hnb74". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:14:56.529982 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.529958 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bcf19dc-706f-4379-a0a1-18029d327224-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5bcf19dc-706f-4379-a0a1-18029d327224" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:14:56.530079 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.530017 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5bcf19dc-706f-4379-a0a1-18029d327224" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:14:56.530123 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.530069 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5bcf19dc-706f-4379-a0a1-18029d327224" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:14:56.535613 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.535588 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bcf19dc-706f-4379-a0a1-18029d327224-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5bcf19dc-706f-4379-a0a1-18029d327224" (UID: "5bcf19dc-706f-4379-a0a1-18029d327224"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 21:14:56.627965 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.627927 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5bcf19dc-706f-4379-a0a1-18029d327224-image-registry-private-configuration\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:14:56.627965 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.627960 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-registry-tls\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:14:56.627965 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.627971 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5bcf19dc-706f-4379-a0a1-18029d327224-installation-pull-secrets\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:14:56.628159 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.627981 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5bcf19dc-706f-4379-a0a1-18029d327224-registry-certificates\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:14:56.628159 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.627989 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnb74\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-kube-api-access-hnb74\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:14:56.628159 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.627998 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5bcf19dc-706f-4379-a0a1-18029d327224-ca-trust-extracted\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:14:56.628159 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.628006 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5bcf19dc-706f-4379-a0a1-18029d327224-trusted-ca\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:14:56.628159 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:56.628014 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5bcf19dc-706f-4379-a0a1-18029d327224-bound-sa-token\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:14:57.067716 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:57.067677 2577 generic.go:356] "Generic (PLEG): container finished" podID="5bcf19dc-706f-4379-a0a1-18029d327224" containerID="cf0dd3d8c371e3c2451cad99c57569ad93aaef12770186b48da98ce6049dce6c" exitCode=0 Jul 06 21:14:57.067870 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:57.067726 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" Jul 06 21:14:57.067870 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:57.067774 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" event={"ID":"5bcf19dc-706f-4379-a0a1-18029d327224","Type":"ContainerDied","Data":"cf0dd3d8c371e3c2451cad99c57569ad93aaef12770186b48da98ce6049dce6c"} Jul 06 21:14:57.067870 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:57.067839 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dddcc6c5c-nmgkt" event={"ID":"5bcf19dc-706f-4379-a0a1-18029d327224","Type":"ContainerDied","Data":"68049bc9cf788eb022f9874ccd1302b672f94207110519a7ae54319e46413c43"} Jul 06 21:14:57.067870 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:57.067860 2577 scope.go:117] "RemoveContainer" containerID="cf0dd3d8c371e3c2451cad99c57569ad93aaef12770186b48da98ce6049dce6c" Jul 06 21:14:57.076660 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:57.076643 2577 scope.go:117] "RemoveContainer" containerID="cf0dd3d8c371e3c2451cad99c57569ad93aaef12770186b48da98ce6049dce6c" Jul 06 21:14:57.076945 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:14:57.076922 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf0dd3d8c371e3c2451cad99c57569ad93aaef12770186b48da98ce6049dce6c\": container with ID starting with cf0dd3d8c371e3c2451cad99c57569ad93aaef12770186b48da98ce6049dce6c not found: ID does not exist" containerID="cf0dd3d8c371e3c2451cad99c57569ad93aaef12770186b48da98ce6049dce6c" Jul 06 21:14:57.077024 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:57.076951 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf0dd3d8c371e3c2451cad99c57569ad93aaef12770186b48da98ce6049dce6c"} err="failed to get container status \"cf0dd3d8c371e3c2451cad99c57569ad93aaef12770186b48da98ce6049dce6c\": rpc error: code = NotFound desc = could not find container \"cf0dd3d8c371e3c2451cad99c57569ad93aaef12770186b48da98ce6049dce6c\": container with ID starting with cf0dd3d8c371e3c2451cad99c57569ad93aaef12770186b48da98ce6049dce6c not found: ID does not exist" Jul 06 21:14:57.088025 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:57.088001 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5dddcc6c5c-nmgkt"] Jul 06 21:14:57.093612 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:57.093590 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5dddcc6c5c-nmgkt"] Jul 06 21:14:58.134435 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:58.134395 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:14:58.488451 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:14:58.488373 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bcf19dc-706f-4379-a0a1-18029d327224" path="/var/lib/kubelet/pods/5bcf19dc-706f-4379-a0a1-18029d327224/volumes" Jul 06 21:15:42.230015 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:15:42.229975 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:15:42.336531 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:15:42.336494 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:15:51.648034 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:15:51.647938 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:15:51.748241 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:15:51.748203 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:15:57.466636 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:15:57.466591 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:15:57.551995 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:15:57.551955 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:08.879283 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:08.879243 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:08.979799 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:08.979763 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:09.080237 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:09.080202 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:09.531737 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:09.531705 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:09.631664 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:09.631627 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:24.634364 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:24.634328 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:24.732916 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:24.732879 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:29.555771 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:29.555732 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:29.645040 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:29.645001 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:42.681216 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:42.681164 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:42.741327 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:42.741297 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:46.187513 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.187448 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:46.209372 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.209329 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm"] Jul 06 21:16:46.209620 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.209608 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bcf19dc-706f-4379-a0a1-18029d327224" containerName="registry" Jul 06 21:16:46.209664 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.209623 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bcf19dc-706f-4379-a0a1-18029d327224" containerName="registry" Jul 06 21:16:46.209701 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.209670 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bcf19dc-706f-4379-a0a1-18029d327224" containerName="registry" Jul 06 21:16:46.212588 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.212572 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm" Jul 06 21:16:46.216161 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.216136 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 21:16:46.216293 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.216158 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 21:16:46.216293 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.216190 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-85jsc\"" Jul 06 21:16:46.216293 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.216190 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 21:16:46.216293 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.216255 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 21:16:46.221937 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.221915 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm"] Jul 06 21:16:46.290079 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.290031 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:46.303093 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.303066 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd84n\" (UniqueName: \"kubernetes.io/projected/a851e7fc-f285-4fde-a4cd-cd1a91b3233d-kube-api-access-vd84n\") pod \"managed-serviceaccount-addon-agent-7869896778-m82bm\" (UID: \"a851e7fc-f285-4fde-a4cd-cd1a91b3233d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm" Jul 06 21:16:46.303220 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.303106 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a851e7fc-f285-4fde-a4cd-cd1a91b3233d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7869896778-m82bm\" (UID: \"a851e7fc-f285-4fde-a4cd-cd1a91b3233d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm" Jul 06 21:16:46.360946 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.360909 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:46.403192 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.403155 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q"] Jul 06 21:16:46.404096 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.404069 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd84n\" (UniqueName: \"kubernetes.io/projected/a851e7fc-f285-4fde-a4cd-cd1a91b3233d-kube-api-access-vd84n\") pod \"managed-serviceaccount-addon-agent-7869896778-m82bm\" (UID: \"a851e7fc-f285-4fde-a4cd-cd1a91b3233d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm" Jul 06 21:16:46.404203 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.404107 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a851e7fc-f285-4fde-a4cd-cd1a91b3233d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7869896778-m82bm\" (UID: \"a851e7fc-f285-4fde-a4cd-cd1a91b3233d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm" Jul 06 21:16:46.406384 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.406365 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" Jul 06 21:16:46.406500 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.406485 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a851e7fc-f285-4fde-a4cd-cd1a91b3233d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7869896778-m82bm\" (UID: \"a851e7fc-f285-4fde-a4cd-cd1a91b3233d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm" Jul 06 21:16:46.409365 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.409345 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 06 21:16:46.412905 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.412883 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd84n\" (UniqueName: \"kubernetes.io/projected/a851e7fc-f285-4fde-a4cd-cd1a91b3233d-kube-api-access-vd84n\") pod \"managed-serviceaccount-addon-agent-7869896778-m82bm\" (UID: \"a851e7fc-f285-4fde-a4cd-cd1a91b3233d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm" Jul 06 21:16:46.417403 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.417379 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q"] Jul 06 21:16:46.507672 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.507590 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/039f711a-43e1-4ff1-992f-41d468409c12-klusterlet-config\") pod \"klusterlet-addon-workmgr-78957cfb8b-n2c7q\" (UID: \"039f711a-43e1-4ff1-992f-41d468409c12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" Jul 06 21:16:46.507815 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.507692 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfh2f\" (UniqueName: \"kubernetes.io/projected/039f711a-43e1-4ff1-992f-41d468409c12-kube-api-access-wfh2f\") pod \"klusterlet-addon-workmgr-78957cfb8b-n2c7q\" (UID: \"039f711a-43e1-4ff1-992f-41d468409c12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" Jul 06 21:16:46.507815 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.507724 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/039f711a-43e1-4ff1-992f-41d468409c12-tmp\") pod \"klusterlet-addon-workmgr-78957cfb8b-n2c7q\" (UID: \"039f711a-43e1-4ff1-992f-41d468409c12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" Jul 06 21:16:46.527423 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.527377 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm" Jul 06 21:16:46.609015 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.608978 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfh2f\" (UniqueName: \"kubernetes.io/projected/039f711a-43e1-4ff1-992f-41d468409c12-kube-api-access-wfh2f\") pod \"klusterlet-addon-workmgr-78957cfb8b-n2c7q\" (UID: \"039f711a-43e1-4ff1-992f-41d468409c12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" Jul 06 21:16:46.609126 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.609035 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/039f711a-43e1-4ff1-992f-41d468409c12-tmp\") pod \"klusterlet-addon-workmgr-78957cfb8b-n2c7q\" (UID: \"039f711a-43e1-4ff1-992f-41d468409c12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" Jul 06 21:16:46.609126 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.609104 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/039f711a-43e1-4ff1-992f-41d468409c12-klusterlet-config\") pod \"klusterlet-addon-workmgr-78957cfb8b-n2c7q\" (UID: \"039f711a-43e1-4ff1-992f-41d468409c12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" Jul 06 21:16:46.610330 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.610281 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/039f711a-43e1-4ff1-992f-41d468409c12-tmp\") pod \"klusterlet-addon-workmgr-78957cfb8b-n2c7q\" (UID: \"039f711a-43e1-4ff1-992f-41d468409c12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" Jul 06 21:16:46.612750 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.612726 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/039f711a-43e1-4ff1-992f-41d468409c12-klusterlet-config\") pod \"klusterlet-addon-workmgr-78957cfb8b-n2c7q\" (UID: \"039f711a-43e1-4ff1-992f-41d468409c12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" Jul 06 21:16:46.617741 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.617702 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfh2f\" (UniqueName: \"kubernetes.io/projected/039f711a-43e1-4ff1-992f-41d468409c12-kube-api-access-wfh2f\") pod \"klusterlet-addon-workmgr-78957cfb8b-n2c7q\" (UID: \"039f711a-43e1-4ff1-992f-41d468409c12\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" Jul 06 21:16:46.653371 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.653344 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm"] Jul 06 21:16:46.655693 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:16:46.655667 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda851e7fc_f285_4fde_a4cd_cd1a91b3233d.slice/crio-8662872ad0f5fa3076e35b28cdb58b5703446c3a0371011abd7b863861bc6301 WatchSource:0}: Error finding container 8662872ad0f5fa3076e35b28cdb58b5703446c3a0371011abd7b863861bc6301: Status 404 returned error can't find the container with id 8662872ad0f5fa3076e35b28cdb58b5703446c3a0371011abd7b863861bc6301 Jul 06 21:16:46.726150 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.726113 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" Jul 06 21:16:46.849537 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:46.849513 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q"] Jul 06 21:16:46.851808 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:16:46.851781 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039f711a_43e1_4ff1_992f_41d468409c12.slice/crio-307c9372457d5a2fe7745c64de61c773c7dd2d4824107f6e18859a82dd3d3399 WatchSource:0}: Error finding container 307c9372457d5a2fe7745c64de61c773c7dd2d4824107f6e18859a82dd3d3399: Status 404 returned error can't find the container with id 307c9372457d5a2fe7745c64de61c773c7dd2d4824107f6e18859a82dd3d3399 Jul 06 21:16:47.474083 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:47.474045 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" event={"ID":"039f711a-43e1-4ff1-992f-41d468409c12","Type":"ContainerStarted","Data":"307c9372457d5a2fe7745c64de61c773c7dd2d4824107f6e18859a82dd3d3399"} Jul 06 21:16:47.475263 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:47.475233 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm" event={"ID":"a851e7fc-f285-4fde-a4cd-cd1a91b3233d","Type":"ContainerStarted","Data":"8662872ad0f5fa3076e35b28cdb58b5703446c3a0371011abd7b863861bc6301"} Jul 06 21:16:51.492668 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:51.492597 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm" event={"ID":"a851e7fc-f285-4fde-a4cd-cd1a91b3233d","Type":"ContainerStarted","Data":"50c549d45b7c81829a89a95bab2fdcafcf7e0c89107d20133641a06065ff1fca"} Jul 06 21:16:51.493995 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:51.493970 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" event={"ID":"039f711a-43e1-4ff1-992f-41d468409c12","Type":"ContainerStarted","Data":"785426d5c1ef02659f89cc2a400dd3b6e6d1db7b58ce410a37cff939e8163a2a"} Jul 06 21:16:51.494071 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:51.494016 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" Jul 06 21:16:51.495867 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:51.495844 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" Jul 06 21:16:51.507918 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:51.507871 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7869896778-m82bm" podStartSLOduration=1.1138006 podStartE2EDuration="5.507857314s" totalImagesPullingTime="4.394056714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:16:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:16:51.507543238 +0000 UTC m=+235.632524662" watchObservedRunningTime="2026-07-06 21:16:51.507857314 +0000 UTC m=+235.632838737" Jul 06 21:16:51.521875 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:51.521819 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78957cfb8b-n2c7q" podStartSLOduration=1.318895922 podStartE2EDuration="5.521802786s" totalImagesPullingTime="4.202906864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:16:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:16:51.521744407 +0000 UTC m=+235.646725831" watchObservedRunningTime="2026-07-06 21:16:51.521802786 +0000 UTC m=+235.646784210" Jul 06 21:16:56.789829 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:56.789795 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:56.846122 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:56.846051 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:57.139817 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:57.139776 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:57.242101 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:57.242067 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:57.313960 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:57.313930 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:16:58.255640 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.255603 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw"] Jul 06 21:16:58.259056 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.259033 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" Jul 06 21:16:58.262003 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.261980 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 21:16:58.262166 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.262148 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 21:16:58.263065 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.263049 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pgb9j\"" Jul 06 21:16:58.271869 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.271825 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw"] Jul 06 21:16:58.300830 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.300795 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ecb68eb-09cb-4ed0-980b-4b416819a26b-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw\" (UID: \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" Jul 06 21:16:58.301006 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.300859 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ecb68eb-09cb-4ed0-980b-4b416819a26b-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw\" (UID: \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" Jul 06 21:16:58.301006 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.300883 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh4q6\" (UniqueName: \"kubernetes.io/projected/9ecb68eb-09cb-4ed0-980b-4b416819a26b-kube-api-access-sh4q6\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw\" (UID: \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" Jul 06 21:16:58.402137 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.402097 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ecb68eb-09cb-4ed0-980b-4b416819a26b-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw\" (UID: \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" Jul 06 21:16:58.402137 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.402138 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh4q6\" (UniqueName: \"kubernetes.io/projected/9ecb68eb-09cb-4ed0-980b-4b416819a26b-kube-api-access-sh4q6\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw\" (UID: \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" Jul 06 21:16:58.402348 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.402182 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ecb68eb-09cb-4ed0-980b-4b416819a26b-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw\" (UID: \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" Jul 06 21:16:58.402600 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.402580 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ecb68eb-09cb-4ed0-980b-4b416819a26b-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw\" (UID: \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" Jul 06 21:16:58.402650 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.402594 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ecb68eb-09cb-4ed0-980b-4b416819a26b-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw\" (UID: \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" Jul 06 21:16:58.411074 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.411047 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh4q6\" (UniqueName: \"kubernetes.io/projected/9ecb68eb-09cb-4ed0-980b-4b416819a26b-kube-api-access-sh4q6\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw\" (UID: \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" Jul 06 21:16:58.568118 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.568033 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" Jul 06 21:16:58.694787 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:58.694747 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw"] Jul 06 21:16:58.697250 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:16:58.697223 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ecb68eb_09cb_4ed0_980b_4b416819a26b.slice/crio-af6d8b631828e9c7ef20710f66f01bad0ecde98795c28b4b3ea6e965c332faa6 WatchSource:0}: Error finding container af6d8b631828e9c7ef20710f66f01bad0ecde98795c28b4b3ea6e965c332faa6: Status 404 returned error can't find the container with id af6d8b631828e9c7ef20710f66f01bad0ecde98795c28b4b3ea6e965c332faa6 Jul 06 21:16:59.521624 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:16:59.521584 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" event={"ID":"9ecb68eb-09cb-4ed0-980b-4b416819a26b","Type":"ContainerStarted","Data":"af6d8b631828e9c7ef20710f66f01bad0ecde98795c28b4b3ea6e965c332faa6"} Jul 06 21:17:06.548618 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:06.548584 2577 generic.go:356] "Generic (PLEG): container finished" podID="9ecb68eb-09cb-4ed0-980b-4b416819a26b" containerID="7f003b5f414285b1d1f15795487adeb2ae39742cfa423569dfaf3527847635fb" exitCode=0 Jul 06 21:17:06.549017 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:06.548678 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" event={"ID":"9ecb68eb-09cb-4ed0-980b-4b416819a26b","Type":"ContainerDied","Data":"7f003b5f414285b1d1f15795487adeb2ae39742cfa423569dfaf3527847635fb"} Jul 06 21:17:09.562172 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:09.562135 2577 generic.go:356] "Generic (PLEG): container finished" podID="9ecb68eb-09cb-4ed0-980b-4b416819a26b" containerID="2f2f0e3140f0c913ae6b6a9bdad967817001386315d603b855c6fbdbe326cd92" exitCode=0 Jul 06 21:17:09.562636 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:09.562222 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" event={"ID":"9ecb68eb-09cb-4ed0-980b-4b416819a26b","Type":"ContainerDied","Data":"2f2f0e3140f0c913ae6b6a9bdad967817001386315d603b855c6fbdbe326cd92"} Jul 06 21:17:15.588011 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:15.587972 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" event={"ID":"9ecb68eb-09cb-4ed0-980b-4b416819a26b","Type":"ContainerStarted","Data":"7ccff1d00c5fc139825dc77bf0535facca3f3be86d187d7caef939e0cd989af9"} Jul 06 21:17:15.632015 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:15.631963 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" podStartSLOduration=2.623400186 podStartE2EDuration="17.631946444s" totalImagesPullingTime="15.008546258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:16:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:17:15.629721445 +0000 UTC m=+259.754702868" watchObservedRunningTime="2026-07-06 21:17:15.631946444 +0000 UTC m=+259.756927867" Jul 06 21:17:16.593607 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:16.593570 2577 generic.go:356] "Generic (PLEG): container finished" podID="9ecb68eb-09cb-4ed0-980b-4b416819a26b" containerID="7ccff1d00c5fc139825dc77bf0535facca3f3be86d187d7caef939e0cd989af9" exitCode=0 Jul 06 21:17:16.593979 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:16.593651 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" event={"ID":"9ecb68eb-09cb-4ed0-980b-4b416819a26b","Type":"ContainerDied","Data":"7ccff1d00c5fc139825dc77bf0535facca3f3be86d187d7caef939e0cd989af9"} Jul 06 21:17:17.725584 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:17.725559 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" Jul 06 21:17:17.856961 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:17.856884 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sh4q6\" (UniqueName: \"kubernetes.io/projected/9ecb68eb-09cb-4ed0-980b-4b416819a26b-kube-api-access-sh4q6\") pod \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\" (UID: \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\") " Jul 06 21:17:17.857106 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:17.856974 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ecb68eb-09cb-4ed0-980b-4b416819a26b-util\") pod \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\" (UID: \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\") " Jul 06 21:17:17.857106 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:17.856998 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ecb68eb-09cb-4ed0-980b-4b416819a26b-bundle\") pod \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\" (UID: \"9ecb68eb-09cb-4ed0-980b-4b416819a26b\") " Jul 06 21:17:17.857661 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:17.857634 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ecb68eb-09cb-4ed0-980b-4b416819a26b-bundle" (OuterVolumeSpecName: "bundle") pod "9ecb68eb-09cb-4ed0-980b-4b416819a26b" (UID: "9ecb68eb-09cb-4ed0-980b-4b416819a26b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 21:17:17.859014 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:17.858990 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ecb68eb-09cb-4ed0-980b-4b416819a26b-kube-api-access-sh4q6" (OuterVolumeSpecName: "kube-api-access-sh4q6") pod "9ecb68eb-09cb-4ed0-980b-4b416819a26b" (UID: "9ecb68eb-09cb-4ed0-980b-4b416819a26b"). InnerVolumeSpecName "kube-api-access-sh4q6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:17:17.861204 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:17.861181 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ecb68eb-09cb-4ed0-980b-4b416819a26b-util" (OuterVolumeSpecName: "util") pod "9ecb68eb-09cb-4ed0-980b-4b416819a26b" (UID: "9ecb68eb-09cb-4ed0-980b-4b416819a26b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 21:17:17.958270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:17.958235 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sh4q6\" (UniqueName: \"kubernetes.io/projected/9ecb68eb-09cb-4ed0-980b-4b416819a26b-kube-api-access-sh4q6\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:17:17.958270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:17.958264 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ecb68eb-09cb-4ed0-980b-4b416819a26b-util\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:17:17.958270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:17.958274 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ecb68eb-09cb-4ed0-980b-4b416819a26b-bundle\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:17:18.603769 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:18.603738 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" Jul 06 21:17:18.603769 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:18.603752 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589tckfw" event={"ID":"9ecb68eb-09cb-4ed0-980b-4b416819a26b","Type":"ContainerDied","Data":"af6d8b631828e9c7ef20710f66f01bad0ecde98795c28b4b3ea6e965c332faa6"} Jul 06 21:17:18.603999 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:18.603795 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af6d8b631828e9c7ef20710f66f01bad0ecde98795c28b4b3ea6e965c332faa6" Jul 06 21:17:19.459108 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:19.459065 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:19.583712 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:19.583680 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:20.736569 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.736534 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr"] Jul 06 21:17:20.736928 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.736826 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ecb68eb-09cb-4ed0-980b-4b416819a26b" containerName="util" Jul 06 21:17:20.736928 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.736840 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ecb68eb-09cb-4ed0-980b-4b416819a26b" containerName="util" Jul 06 21:17:20.736928 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.736858 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ecb68eb-09cb-4ed0-980b-4b416819a26b" containerName="extract" Jul 06 21:17:20.736928 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.736867 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ecb68eb-09cb-4ed0-980b-4b416819a26b" containerName="extract" Jul 06 21:17:20.736928 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.736885 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ecb68eb-09cb-4ed0-980b-4b416819a26b" containerName="pull" Jul 06 21:17:20.736928 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.736891 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ecb68eb-09cb-4ed0-980b-4b416819a26b" containerName="pull" Jul 06 21:17:20.737116 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.736932 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ecb68eb-09cb-4ed0-980b-4b416819a26b" containerName="extract" Jul 06 21:17:20.782406 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.782370 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr"] Jul 06 21:17:20.782573 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.782493 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr" Jul 06 21:17:20.790003 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.789972 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 06 21:17:20.790283 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.790261 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-4ccvb\"" Jul 06 21:17:20.790440 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.790420 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 06 21:17:20.791584 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.791567 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 06 21:17:20.880166 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.880126 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/d792bbcb-2532-4c50-aaa0-134c6a6ea94b-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr\" (UID: \"d792bbcb-2532-4c50-aaa0-134c6a6ea94b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr" Jul 06 21:17:20.880166 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.880166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdctv\" (UniqueName: \"kubernetes.io/projected/d792bbcb-2532-4c50-aaa0-134c6a6ea94b-kube-api-access-jdctv\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr\" (UID: \"d792bbcb-2532-4c50-aaa0-134c6a6ea94b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr" Jul 06 21:17:20.981100 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.981064 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/d792bbcb-2532-4c50-aaa0-134c6a6ea94b-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr\" (UID: \"d792bbcb-2532-4c50-aaa0-134c6a6ea94b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr" Jul 06 21:17:20.981100 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.981099 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdctv\" (UniqueName: \"kubernetes.io/projected/d792bbcb-2532-4c50-aaa0-134c6a6ea94b-kube-api-access-jdctv\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr\" (UID: \"d792bbcb-2532-4c50-aaa0-134c6a6ea94b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr" Jul 06 21:17:20.983457 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.983430 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/d792bbcb-2532-4c50-aaa0-134c6a6ea94b-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr\" (UID: \"d792bbcb-2532-4c50-aaa0-134c6a6ea94b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr" Jul 06 21:17:20.990571 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:20.990519 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdctv\" (UniqueName: \"kubernetes.io/projected/d792bbcb-2532-4c50-aaa0-134c6a6ea94b-kube-api-access-jdctv\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr\" (UID: \"d792bbcb-2532-4c50-aaa0-134c6a6ea94b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr" Jul 06 21:17:21.092425 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:21.092394 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr" Jul 06 21:17:21.242964 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:21.242940 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr"] Jul 06 21:17:21.245645 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:17:21.245609 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd792bbcb_2532_4c50_aaa0_134c6a6ea94b.slice/crio-57f97c6e1acf7455888f59fe5afd6da81bc346d29307b1188ee9835873d6f912 WatchSource:0}: Error finding container 57f97c6e1acf7455888f59fe5afd6da81bc346d29307b1188ee9835873d6f912: Status 404 returned error can't find the container with id 57f97c6e1acf7455888f59fe5afd6da81bc346d29307b1188ee9835873d6f912 Jul 06 21:17:21.616191 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:21.616154 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr" event={"ID":"d792bbcb-2532-4c50-aaa0-134c6a6ea94b","Type":"ContainerStarted","Data":"57f97c6e1acf7455888f59fe5afd6da81bc346d29307b1188ee9835873d6f912"} Jul 06 21:17:23.514972 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:23.514933 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:23.585205 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:23.585168 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:25.633136 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:25.633039 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr" event={"ID":"d792bbcb-2532-4c50-aaa0-134c6a6ea94b","Type":"ContainerStarted","Data":"00128c7929bc114b9d1373b73afe21f572d4c9ee8206866fd9c94bd34465cf51"} Jul 06 21:17:25.633617 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:25.633149 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr" Jul 06 21:17:25.669653 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:25.669588 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr" podStartSLOduration=1.558628008 podStartE2EDuration="5.669567691s" totalImagesPullingTime="4.110939683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:17:25.669383723 +0000 UTC m=+269.794365145" watchObservedRunningTime="2026-07-06 21:17:25.669567691 +0000 UTC m=+269.794549115" Jul 06 21:17:25.794230 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:25.794196 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:26.135617 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.135584 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:26.150815 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.150785 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-p6l7f"] Jul 06 21:17:26.152950 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.152925 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:26.156745 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.156719 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 06 21:17:26.156851 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.156769 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jul 06 21:17:26.157354 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.157339 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-5xc89\"" Jul 06 21:17:26.172351 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.172324 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-p6l7f"] Jul 06 21:17:26.224487 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.224425 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d8573091-c228-4391-bdf1-d8ef4cbff70a-certificates\") pod \"keda-operator-68bff59c-p6l7f\" (UID: \"d8573091-c228-4391-bdf1-d8ef4cbff70a\") " pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:26.224487 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.224489 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhxqf\" (UniqueName: \"kubernetes.io/projected/d8573091-c228-4391-bdf1-d8ef4cbff70a-kube-api-access-vhxqf\") pod \"keda-operator-68bff59c-p6l7f\" (UID: \"d8573091-c228-4391-bdf1-d8ef4cbff70a\") " pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:26.224715 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.224589 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/d8573091-c228-4391-bdf1-d8ef4cbff70a-cabundle0\") pod \"keda-operator-68bff59c-p6l7f\" (UID: \"d8573091-c228-4391-bdf1-d8ef4cbff70a\") " pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:26.325814 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.325770 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d8573091-c228-4391-bdf1-d8ef4cbff70a-certificates\") pod \"keda-operator-68bff59c-p6l7f\" (UID: \"d8573091-c228-4391-bdf1-d8ef4cbff70a\") " pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:26.325814 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.325816 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhxqf\" (UniqueName: \"kubernetes.io/projected/d8573091-c228-4391-bdf1-d8ef4cbff70a-kube-api-access-vhxqf\") pod \"keda-operator-68bff59c-p6l7f\" (UID: \"d8573091-c228-4391-bdf1-d8ef4cbff70a\") " pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:26.326064 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.325861 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/d8573091-c228-4391-bdf1-d8ef4cbff70a-cabundle0\") pod \"keda-operator-68bff59c-p6l7f\" (UID: \"d8573091-c228-4391-bdf1-d8ef4cbff70a\") " pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:26.326064 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:17:26.325918 2577 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found Jul 06 21:17:26.326064 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:17:26.325944 2577 secret.go:281] references non-existent secret key: ca.crt Jul 06 21:17:26.326064 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:17:26.325952 2577 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 06 21:17:26.326064 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:17:26.325963 2577 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-p6l7f: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Jul 06 21:17:26.326064 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:17:26.326014 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8573091-c228-4391-bdf1-d8ef4cbff70a-certificates podName:d8573091-c228-4391-bdf1-d8ef4cbff70a nodeName:}" failed. No retries permitted until 2026-07-06 21:17:26.825998801 +0000 UTC m=+270.950980201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/d8573091-c228-4391-bdf1-d8ef4cbff70a-certificates") pod "keda-operator-68bff59c-p6l7f" (UID: "d8573091-c228-4391-bdf1-d8ef4cbff70a") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Jul 06 21:17:26.326649 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.326627 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/d8573091-c228-4391-bdf1-d8ef4cbff70a-cabundle0\") pod \"keda-operator-68bff59c-p6l7f\" (UID: \"d8573091-c228-4391-bdf1-d8ef4cbff70a\") " pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:26.336763 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.336736 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhxqf\" (UniqueName: \"kubernetes.io/projected/d8573091-c228-4391-bdf1-d8ef4cbff70a-kube-api-access-vhxqf\") pod \"keda-operator-68bff59c-p6l7f\" (UID: \"d8573091-c228-4391-bdf1-d8ef4cbff70a\") " pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:26.589342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.589303 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:26.734641 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.734603 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:26.798521 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.798488 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-96tk4"] Jul 06 21:17:26.800792 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.800766 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:26.803962 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.803939 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jul 06 21:17:26.812988 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.812961 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-96tk4"] Jul 06 21:17:26.820799 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.820769 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:26.830076 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.830047 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d8573091-c228-4391-bdf1-d8ef4cbff70a-certificates\") pod \"keda-operator-68bff59c-p6l7f\" (UID: \"d8573091-c228-4391-bdf1-d8ef4cbff70a\") " pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:26.830260 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:17:26.830243 2577 secret.go:281] references non-existent secret key: ca.crt Jul 06 21:17:26.830338 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:17:26.830267 2577 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 06 21:17:26.830338 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:17:26.830278 2577 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-p6l7f: references non-existent secret key: ca.crt Jul 06 21:17:26.830416 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:17:26.830340 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8573091-c228-4391-bdf1-d8ef4cbff70a-certificates podName:d8573091-c228-4391-bdf1-d8ef4cbff70a nodeName:}" failed. No retries permitted until 2026-07-06 21:17:27.830321651 +0000 UTC m=+271.955303064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/d8573091-c228-4391-bdf1-d8ef4cbff70a-certificates") pod "keda-operator-68bff59c-p6l7f" (UID: "d8573091-c228-4391-bdf1-d8ef4cbff70a") : references non-existent secret key: ca.crt Jul 06 21:17:26.930504 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.930420 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/46938848-ba74-4440-a461-4e1cce4c783d-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-96tk4\" (UID: \"46938848-ba74-4440-a461-4e1cce4c783d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:26.930504 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.930448 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/46938848-ba74-4440-a461-4e1cce4c783d-certificates\") pod \"keda-metrics-apiserver-559f75f947-96tk4\" (UID: \"46938848-ba74-4440-a461-4e1cce4c783d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:26.930504 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:26.930493 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95qbp\" (UniqueName: \"kubernetes.io/projected/46938848-ba74-4440-a461-4e1cce4c783d-kube-api-access-95qbp\") pod \"keda-metrics-apiserver-559f75f947-96tk4\" (UID: \"46938848-ba74-4440-a461-4e1cce4c783d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:27.031866 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.031816 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95qbp\" (UniqueName: \"kubernetes.io/projected/46938848-ba74-4440-a461-4e1cce4c783d-kube-api-access-95qbp\") pod \"keda-metrics-apiserver-559f75f947-96tk4\" (UID: \"46938848-ba74-4440-a461-4e1cce4c783d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:27.032052 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.031944 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/46938848-ba74-4440-a461-4e1cce4c783d-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-96tk4\" (UID: \"46938848-ba74-4440-a461-4e1cce4c783d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:27.032052 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.031971 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/46938848-ba74-4440-a461-4e1cce4c783d-certificates\") pod \"keda-metrics-apiserver-559f75f947-96tk4\" (UID: \"46938848-ba74-4440-a461-4e1cce4c783d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:27.032181 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:17:27.032152 2577 secret.go:281] references non-existent secret key: tls.crt Jul 06 21:17:27.032211 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:17:27.032187 2577 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Jul 06 21:17:27.032211 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:17:27.032208 2577 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-559f75f947-96tk4: references non-existent secret key: tls.crt Jul 06 21:17:27.032282 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:17:27.032273 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46938848-ba74-4440-a461-4e1cce4c783d-certificates podName:46938848-ba74-4440-a461-4e1cce4c783d nodeName:}" failed. No retries permitted until 2026-07-06 21:17:27.532256124 +0000 UTC m=+271.657237535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/46938848-ba74-4440-a461-4e1cce4c783d-certificates") pod "keda-metrics-apiserver-559f75f947-96tk4" (UID: "46938848-ba74-4440-a461-4e1cce4c783d") : references non-existent secret key: tls.crt Jul 06 21:17:27.032323 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.032304 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/46938848-ba74-4440-a461-4e1cce4c783d-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-96tk4\" (UID: \"46938848-ba74-4440-a461-4e1cce4c783d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:27.041395 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.041371 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95qbp\" (UniqueName: \"kubernetes.io/projected/46938848-ba74-4440-a461-4e1cce4c783d-kube-api-access-95qbp\") pod \"keda-metrics-apiserver-559f75f947-96tk4\" (UID: \"46938848-ba74-4440-a461-4e1cce4c783d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:27.238895 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.238799 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:27.258964 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.258930 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-hkjwc"] Jul 06 21:17:27.261211 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.261190 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-hkjwc" Jul 06 21:17:27.264561 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.264540 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jul 06 21:17:27.284809 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.284778 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-hkjwc"] Jul 06 21:17:27.333801 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.333771 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b01416f5-b63f-4b0b-9a23-45e6afb647f3-certificates\") pod \"keda-admission-5fbd5c4644-hkjwc\" (UID: \"b01416f5-b63f-4b0b-9a23-45e6afb647f3\") " pod="openshift-keda/keda-admission-5fbd5c4644-hkjwc" Jul 06 21:17:27.333948 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.333820 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqf8b\" (UniqueName: \"kubernetes.io/projected/b01416f5-b63f-4b0b-9a23-45e6afb647f3-kube-api-access-hqf8b\") pod \"keda-admission-5fbd5c4644-hkjwc\" (UID: \"b01416f5-b63f-4b0b-9a23-45e6afb647f3\") " pod="openshift-keda/keda-admission-5fbd5c4644-hkjwc" Jul 06 21:17:27.351956 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.351923 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:27.434644 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.434599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqf8b\" (UniqueName: \"kubernetes.io/projected/b01416f5-b63f-4b0b-9a23-45e6afb647f3-kube-api-access-hqf8b\") pod \"keda-admission-5fbd5c4644-hkjwc\" (UID: \"b01416f5-b63f-4b0b-9a23-45e6afb647f3\") " pod="openshift-keda/keda-admission-5fbd5c4644-hkjwc" Jul 06 21:17:27.434832 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.434731 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b01416f5-b63f-4b0b-9a23-45e6afb647f3-certificates\") pod \"keda-admission-5fbd5c4644-hkjwc\" (UID: \"b01416f5-b63f-4b0b-9a23-45e6afb647f3\") " pod="openshift-keda/keda-admission-5fbd5c4644-hkjwc" Jul 06 21:17:27.437305 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.437279 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b01416f5-b63f-4b0b-9a23-45e6afb647f3-certificates\") pod \"keda-admission-5fbd5c4644-hkjwc\" (UID: \"b01416f5-b63f-4b0b-9a23-45e6afb647f3\") " pod="openshift-keda/keda-admission-5fbd5c4644-hkjwc" Jul 06 21:17:27.446374 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.446352 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqf8b\" (UniqueName: \"kubernetes.io/projected/b01416f5-b63f-4b0b-9a23-45e6afb647f3-kube-api-access-hqf8b\") pod \"keda-admission-5fbd5c4644-hkjwc\" (UID: \"b01416f5-b63f-4b0b-9a23-45e6afb647f3\") " pod="openshift-keda/keda-admission-5fbd5c4644-hkjwc" Jul 06 21:17:27.452440 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.452409 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:27.535733 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.535643 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/46938848-ba74-4440-a461-4e1cce4c783d-certificates\") pod \"keda-metrics-apiserver-559f75f947-96tk4\" (UID: \"46938848-ba74-4440-a461-4e1cce4c783d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:27.538160 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.538132 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/46938848-ba74-4440-a461-4e1cce4c783d-certificates\") pod \"keda-metrics-apiserver-559f75f947-96tk4\" (UID: \"46938848-ba74-4440-a461-4e1cce4c783d\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:27.571666 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.571631 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-hkjwc" Jul 06 21:17:27.653800 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.653763 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:27.702169 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.702141 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-hkjwc"] Jul 06 21:17:27.704688 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:17:27.704660 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb01416f5_b63f_4b0b_9a23_45e6afb647f3.slice/crio-8b8fcc0477b5bfffd59772112d4ac74428a73d38987e41bf27f7bacdd3d52909 WatchSource:0}: Error finding container 8b8fcc0477b5bfffd59772112d4ac74428a73d38987e41bf27f7bacdd3d52909: Status 404 returned error can't find the container with id 8b8fcc0477b5bfffd59772112d4ac74428a73d38987e41bf27f7bacdd3d52909 Jul 06 21:17:27.711283 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.711265 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:27.819185 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.819152 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:27.838105 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.838077 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d8573091-c228-4391-bdf1-d8ef4cbff70a-certificates\") pod \"keda-operator-68bff59c-p6l7f\" (UID: \"d8573091-c228-4391-bdf1-d8ef4cbff70a\") " pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:27.840403 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.840383 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d8573091-c228-4391-bdf1-d8ef4cbff70a-certificates\") pod \"keda-operator-68bff59c-p6l7f\" (UID: \"d8573091-c228-4391-bdf1-d8ef4cbff70a\") " pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:27.859306 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.859266 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-96tk4"] Jul 06 21:17:27.862824 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:17:27.862798 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46938848_ba74_4440_a461_4e1cce4c783d.slice/crio-042e4bab3a6c0a3dfa4319268f5fdb097ee53c2d92de2822f0fab83674e7f4bf WatchSource:0}: Error finding container 042e4bab3a6c0a3dfa4319268f5fdb097ee53c2d92de2822f0fab83674e7f4bf: Status 404 returned error can't find the container with id 042e4bab3a6c0a3dfa4319268f5fdb097ee53c2d92de2822f0fab83674e7f4bf Jul 06 21:17:27.963059 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:27.963029 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:28.017899 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:28.017862 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:28.108895 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:28.108846 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-p6l7f"] Jul 06 21:17:28.111720 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:17:28.111685 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8573091_c228_4391_bdf1_d8ef4cbff70a.slice/crio-6d776662fbc3fdaf48277a12c8cb736d3acec3066513c4f89c429cb14f51b394 WatchSource:0}: Error finding container 6d776662fbc3fdaf48277a12c8cb736d3acec3066513c4f89c429cb14f51b394: Status 404 returned error can't find the container with id 6d776662fbc3fdaf48277a12c8cb736d3acec3066513c4f89c429cb14f51b394 Jul 06 21:17:28.138720 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:28.138673 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:28.647866 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:28.647786 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-hkjwc" event={"ID":"b01416f5-b63f-4b0b-9a23-45e6afb647f3","Type":"ContainerStarted","Data":"8b8fcc0477b5bfffd59772112d4ac74428a73d38987e41bf27f7bacdd3d52909"} Jul 06 21:17:28.649477 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:28.649368 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-p6l7f" event={"ID":"d8573091-c228-4391-bdf1-d8ef4cbff70a","Type":"ContainerStarted","Data":"6d776662fbc3fdaf48277a12c8cb736d3acec3066513c4f89c429cb14f51b394"} Jul 06 21:17:28.650686 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:28.650656 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" event={"ID":"46938848-ba74-4440-a461-4e1cce4c783d","Type":"ContainerStarted","Data":"042e4bab3a6c0a3dfa4319268f5fdb097ee53c2d92de2822f0fab83674e7f4bf"} Jul 06 21:17:29.658400 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:29.658358 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-hkjwc" event={"ID":"b01416f5-b63f-4b0b-9a23-45e6afb647f3","Type":"ContainerStarted","Data":"72ba15c7bc591df371d0cc8f66508415b327ecc684401211c87f21842c370f1f"} Jul 06 21:17:29.658893 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:29.658831 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-hkjwc" Jul 06 21:17:29.686200 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:29.686149 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-hkjwc" podStartSLOduration=1.230558909 podStartE2EDuration="2.686129762s" totalImagesPullingTime="1.455570853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:17:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:17:29.684514478 +0000 UTC m=+273.809495903" watchObservedRunningTime="2026-07-06 21:17:29.686129762 +0000 UTC m=+273.811111187" Jul 06 21:17:31.669247 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:31.669209 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-p6l7f" event={"ID":"d8573091-c228-4391-bdf1-d8ef4cbff70a","Type":"ContainerStarted","Data":"f169a5bddfa209f8bb4f24f11d720d4e76203ed1e0ec95a65005986b96914a89"} Jul 06 21:17:31.669704 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:31.669309 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:31.670814 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:31.670788 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" event={"ID":"46938848-ba74-4440-a461-4e1cce4c783d","Type":"ContainerStarted","Data":"5fc94c2530edb727cc37f36639c88ce1881406b37fe9ac107ad492b0d2d19be2"} Jul 06 21:17:31.670932 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:31.670917 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:31.688596 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:31.688533 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-p6l7f" podStartSLOduration=2.301192052 podStartE2EDuration="5.688511363s" totalImagesPullingTime="3.387319311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:17:31.686428856 +0000 UTC m=+275.811410280" watchObservedRunningTime="2026-07-06 21:17:31.688511363 +0000 UTC m=+275.813492788" Jul 06 21:17:31.704476 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:31.704424 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" podStartSLOduration=2.071457587 podStartE2EDuration="5.704410273s" totalImagesPullingTime="3.632952686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:17:31.703009778 +0000 UTC m=+275.827991229" watchObservedRunningTime="2026-07-06 21:17:31.704410273 +0000 UTC m=+275.829391697" Jul 06 21:17:33.212291 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:33.212253 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:35.770038 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:35.769946 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:35.870846 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:35.870803 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:42.679523 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:42.679488 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-96tk4" Jul 06 21:17:46.639574 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:46.639541 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r6hgr" Jul 06 21:17:50.666099 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:50.666070 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-hkjwc" Jul 06 21:17:50.972997 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:50.972919 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:51.082903 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:51.082866 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:17:52.677313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:52.677285 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-p6l7f" Jul 06 21:17:56.285948 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:56.285923 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:17:56.288091 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:56.288069 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:17:56.290006 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:56.289988 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:17:56.292240 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:56.292217 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:17:56.297018 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:17:56.296992 2577 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 21:18:33.067968 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.067930 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:18:33.130624 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.130592 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7ccfc88547-ftpdr"] Jul 06 21:18:33.132858 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.132842 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" Jul 06 21:18:33.135353 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.135328 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Jul 06 21:18:33.135454 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.135365 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-mmfhx\"" Jul 06 21:18:33.136311 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.136292 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 06 21:18:33.136394 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.136360 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 06 21:18:33.140119 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.140101 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-lvd4m"] Jul 06 21:18:33.142077 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.142058 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-lvd4m" Jul 06 21:18:33.144346 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.144324 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jul 06 21:18:33.144557 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.144544 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-b5rw8\"" Jul 06 21:18:33.149636 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.149618 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7ccfc88547-ftpdr"] Jul 06 21:18:33.154093 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.154071 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-lvd4m"] Jul 06 21:18:33.236133 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.236095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79e9a705-3d7b-446f-85f1-7538ef885f41-cert\") pod \"kserve-controller-manager-7ccfc88547-ftpdr\" (UID: \"79e9a705-3d7b-446f-85f1-7538ef885f41\") " pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" Jul 06 21:18:33.236314 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.236142 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a45f6536-6c65-4922-ba70-19fc52da1d94-cert\") pod \"llmisvc-controller-manager-67d95fdc48-lvd4m\" (UID: \"a45f6536-6c65-4922-ba70-19fc52da1d94\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-lvd4m" Jul 06 21:18:33.236314 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.236221 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf2k2\" (UniqueName: \"kubernetes.io/projected/a45f6536-6c65-4922-ba70-19fc52da1d94-kube-api-access-sf2k2\") pod \"llmisvc-controller-manager-67d95fdc48-lvd4m\" (UID: \"a45f6536-6c65-4922-ba70-19fc52da1d94\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-lvd4m" Jul 06 21:18:33.236314 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.236265 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb9ww\" (UniqueName: \"kubernetes.io/projected/79e9a705-3d7b-446f-85f1-7538ef885f41-kube-api-access-pb9ww\") pod \"kserve-controller-manager-7ccfc88547-ftpdr\" (UID: \"79e9a705-3d7b-446f-85f1-7538ef885f41\") " pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" Jul 06 21:18:33.337157 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.337116 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79e9a705-3d7b-446f-85f1-7538ef885f41-cert\") pod \"kserve-controller-manager-7ccfc88547-ftpdr\" (UID: \"79e9a705-3d7b-446f-85f1-7538ef885f41\") " pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" Jul 06 21:18:33.337157 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.337165 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a45f6536-6c65-4922-ba70-19fc52da1d94-cert\") pod \"llmisvc-controller-manager-67d95fdc48-lvd4m\" (UID: \"a45f6536-6c65-4922-ba70-19fc52da1d94\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-lvd4m" Jul 06 21:18:33.337423 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.337198 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf2k2\" (UniqueName: \"kubernetes.io/projected/a45f6536-6c65-4922-ba70-19fc52da1d94-kube-api-access-sf2k2\") pod \"llmisvc-controller-manager-67d95fdc48-lvd4m\" (UID: \"a45f6536-6c65-4922-ba70-19fc52da1d94\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-lvd4m" Jul 06 21:18:33.337423 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.337222 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb9ww\" (UniqueName: \"kubernetes.io/projected/79e9a705-3d7b-446f-85f1-7538ef885f41-kube-api-access-pb9ww\") pod \"kserve-controller-manager-7ccfc88547-ftpdr\" (UID: \"79e9a705-3d7b-446f-85f1-7538ef885f41\") " pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" Jul 06 21:18:33.337423 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:18:33.337294 2577 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 06 21:18:33.337423 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:18:33.337384 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e9a705-3d7b-446f-85f1-7538ef885f41-cert podName:79e9a705-3d7b-446f-85f1-7538ef885f41 nodeName:}" failed. No retries permitted until 2026-07-06 21:18:33.837361275 +0000 UTC m=+337.962342689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/79e9a705-3d7b-446f-85f1-7538ef885f41-cert") pod "kserve-controller-manager-7ccfc88547-ftpdr" (UID: "79e9a705-3d7b-446f-85f1-7538ef885f41") : secret "kserve-webhook-server-cert" not found Jul 06 21:18:33.339634 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.339616 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a45f6536-6c65-4922-ba70-19fc52da1d94-cert\") pod \"llmisvc-controller-manager-67d95fdc48-lvd4m\" (UID: \"a45f6536-6c65-4922-ba70-19fc52da1d94\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-lvd4m" Jul 06 21:18:33.354326 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.354303 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb9ww\" (UniqueName: \"kubernetes.io/projected/79e9a705-3d7b-446f-85f1-7538ef885f41-kube-api-access-pb9ww\") pod \"kserve-controller-manager-7ccfc88547-ftpdr\" (UID: \"79e9a705-3d7b-446f-85f1-7538ef885f41\") " pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" Jul 06 21:18:33.364285 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.364261 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf2k2\" (UniqueName: \"kubernetes.io/projected/a45f6536-6c65-4922-ba70-19fc52da1d94-kube-api-access-sf2k2\") pod \"llmisvc-controller-manager-67d95fdc48-lvd4m\" (UID: \"a45f6536-6c65-4922-ba70-19fc52da1d94\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-lvd4m" Jul 06 21:18:33.451977 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.451943 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-lvd4m" Jul 06 21:18:33.607901 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.607696 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-lvd4m"] Jul 06 21:18:33.610374 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:18:33.610347 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda45f6536_6c65_4922_ba70_19fc52da1d94.slice/crio-f5b94b43940c47db77e5f6c1cb76c3b359eda1e0d9d0753d57cfe19d398df225 WatchSource:0}: Error finding container f5b94b43940c47db77e5f6c1cb76c3b359eda1e0d9d0753d57cfe19d398df225: Status 404 returned error can't find the container with id f5b94b43940c47db77e5f6c1cb76c3b359eda1e0d9d0753d57cfe19d398df225 Jul 06 21:18:33.611642 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.611620 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 21:18:33.842155 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.842115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79e9a705-3d7b-446f-85f1-7538ef885f41-cert\") pod \"kserve-controller-manager-7ccfc88547-ftpdr\" (UID: \"79e9a705-3d7b-446f-85f1-7538ef885f41\") " pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" Jul 06 21:18:33.844544 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.844522 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79e9a705-3d7b-446f-85f1-7538ef885f41-cert\") pod \"kserve-controller-manager-7ccfc88547-ftpdr\" (UID: \"79e9a705-3d7b-446f-85f1-7538ef885f41\") " pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" Jul 06 21:18:33.925541 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:33.925443 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-lvd4m" event={"ID":"a45f6536-6c65-4922-ba70-19fc52da1d94","Type":"ContainerStarted","Data":"f5b94b43940c47db77e5f6c1cb76c3b359eda1e0d9d0753d57cfe19d398df225"} Jul 06 21:18:34.043194 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:34.043150 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" Jul 06 21:18:34.361747 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:34.361711 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:18:34.394094 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:34.393968 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7ccfc88547-ftpdr"] Jul 06 21:18:34.418905 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:18:34.418870 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e9a705_3d7b_446f_85f1_7538ef885f41.slice/crio-0a1877d1d3763107ed672250dff023f4a623368961c8d257c71df6a9c3cd0c43 WatchSource:0}: Error finding container 0a1877d1d3763107ed672250dff023f4a623368961c8d257c71df6a9c3cd0c43: Status 404 returned error can't find the container with id 0a1877d1d3763107ed672250dff023f4a623368961c8d257c71df6a9c3cd0c43 Jul 06 21:18:34.932324 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:34.932285 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" event={"ID":"79e9a705-3d7b-446f-85f1-7538ef885f41","Type":"ContainerStarted","Data":"0a1877d1d3763107ed672250dff023f4a623368961c8d257c71df6a9c3cd0c43"} Jul 06 21:18:35.941330 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:35.941285 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-lvd4m" event={"ID":"a45f6536-6c65-4922-ba70-19fc52da1d94","Type":"ContainerStarted","Data":"491a477d10bff882815738e73cd25a8ba0e6e717282bb46fea797c4f905acbe2"} Jul 06 21:18:35.941945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:35.941918 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-lvd4m" Jul 06 21:18:35.968194 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:35.968141 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-67d95fdc48-lvd4m" podStartSLOduration=1.247902121 podStartE2EDuration="2.968126165s" totalImagesPullingTime="1.720224044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:18:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:18:35.965006919 +0000 UTC m=+340.089988343" watchObservedRunningTime="2026-07-06 21:18:35.968126165 +0000 UTC m=+340.093107588" Jul 06 21:18:37.951166 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:37.951073 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" event={"ID":"79e9a705-3d7b-446f-85f1-7538ef885f41","Type":"ContainerStarted","Data":"221c00ee049d5cb6c32fcaba3c2f7c3764c717f118f9383b98c503ac95986ba4"} Jul 06 21:18:37.968336 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:37.968289 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" podStartSLOduration=1.798427724 podStartE2EDuration="4.968273189s" totalImagesPullingTime="3.169845465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:18:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:18:37.967057821 +0000 UTC m=+342.092039243" watchObservedRunningTime="2026-07-06 21:18:37.968273189 +0000 UTC m=+342.093254611" Jul 06 21:18:38.955003 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:18:38.954973 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" Jul 06 21:19:07.953432 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:07.953398 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-lvd4m" Jul 06 21:19:09.281418 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.281380 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7ccfc88547-ftpdr"] Jul 06 21:19:09.281840 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.281616 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" podUID="79e9a705-3d7b-446f-85f1-7538ef885f41" containerName="manager" containerID="cri-o://221c00ee049d5cb6c32fcaba3c2f7c3764c717f118f9383b98c503ac95986ba4" gracePeriod=10 Jul 06 21:19:09.286312 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.286290 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" Jul 06 21:19:09.301639 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.301616 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7ccfc88547-nqxz9"] Jul 06 21:19:09.305041 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.305021 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7ccfc88547-nqxz9" Jul 06 21:19:09.315081 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.315046 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7ccfc88547-nqxz9"] Jul 06 21:19:09.422543 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.422508 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkjn6\" (UniqueName: \"kubernetes.io/projected/bfff6677-9f60-42a3-a03e-9dbfdf63e7d3-kube-api-access-tkjn6\") pod \"kserve-controller-manager-7ccfc88547-nqxz9\" (UID: \"bfff6677-9f60-42a3-a03e-9dbfdf63e7d3\") " pod="kserve/kserve-controller-manager-7ccfc88547-nqxz9" Jul 06 21:19:09.422703 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.422585 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bfff6677-9f60-42a3-a03e-9dbfdf63e7d3-cert\") pod \"kserve-controller-manager-7ccfc88547-nqxz9\" (UID: \"bfff6677-9f60-42a3-a03e-9dbfdf63e7d3\") " pod="kserve/kserve-controller-manager-7ccfc88547-nqxz9" Jul 06 21:19:09.521408 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.521383 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" Jul 06 21:19:09.523367 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.523348 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkjn6\" (UniqueName: \"kubernetes.io/projected/bfff6677-9f60-42a3-a03e-9dbfdf63e7d3-kube-api-access-tkjn6\") pod \"kserve-controller-manager-7ccfc88547-nqxz9\" (UID: \"bfff6677-9f60-42a3-a03e-9dbfdf63e7d3\") " pod="kserve/kserve-controller-manager-7ccfc88547-nqxz9" Jul 06 21:19:09.523441 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.523396 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bfff6677-9f60-42a3-a03e-9dbfdf63e7d3-cert\") pod \"kserve-controller-manager-7ccfc88547-nqxz9\" (UID: \"bfff6677-9f60-42a3-a03e-9dbfdf63e7d3\") " pod="kserve/kserve-controller-manager-7ccfc88547-nqxz9" Jul 06 21:19:09.525521 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.525499 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bfff6677-9f60-42a3-a03e-9dbfdf63e7d3-cert\") pod \"kserve-controller-manager-7ccfc88547-nqxz9\" (UID: \"bfff6677-9f60-42a3-a03e-9dbfdf63e7d3\") " pod="kserve/kserve-controller-manager-7ccfc88547-nqxz9" Jul 06 21:19:09.531940 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.531875 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkjn6\" (UniqueName: \"kubernetes.io/projected/bfff6677-9f60-42a3-a03e-9dbfdf63e7d3-kube-api-access-tkjn6\") pod \"kserve-controller-manager-7ccfc88547-nqxz9\" (UID: \"bfff6677-9f60-42a3-a03e-9dbfdf63e7d3\") " pod="kserve/kserve-controller-manager-7ccfc88547-nqxz9" Jul 06 21:19:09.624758 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.624722 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pb9ww\" (UniqueName: \"kubernetes.io/projected/79e9a705-3d7b-446f-85f1-7538ef885f41-kube-api-access-pb9ww\") pod \"79e9a705-3d7b-446f-85f1-7538ef885f41\" (UID: \"79e9a705-3d7b-446f-85f1-7538ef885f41\") " Jul 06 21:19:09.624932 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.624853 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79e9a705-3d7b-446f-85f1-7538ef885f41-cert\") pod \"79e9a705-3d7b-446f-85f1-7538ef885f41\" (UID: \"79e9a705-3d7b-446f-85f1-7538ef885f41\") " Jul 06 21:19:09.626926 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.626894 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79e9a705-3d7b-446f-85f1-7538ef885f41-cert" (OuterVolumeSpecName: "cert") pod "79e9a705-3d7b-446f-85f1-7538ef885f41" (UID: "79e9a705-3d7b-446f-85f1-7538ef885f41"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:19:09.627016 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.626943 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79e9a705-3d7b-446f-85f1-7538ef885f41-kube-api-access-pb9ww" (OuterVolumeSpecName: "kube-api-access-pb9ww") pod "79e9a705-3d7b-446f-85f1-7538ef885f41" (UID: "79e9a705-3d7b-446f-85f1-7538ef885f41"). InnerVolumeSpecName "kube-api-access-pb9ww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:19:09.657661 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.657638 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7ccfc88547-nqxz9" Jul 06 21:19:09.725563 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.725533 2577 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79e9a705-3d7b-446f-85f1-7538ef885f41-cert\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:19:09.725563 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.725561 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pb9ww\" (UniqueName: \"kubernetes.io/projected/79e9a705-3d7b-446f-85f1-7538ef885f41-kube-api-access-pb9ww\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:19:09.785244 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:09.785218 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7ccfc88547-nqxz9"] Jul 06 21:19:09.787569 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:19:09.787531 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfff6677_9f60_42a3_a03e_9dbfdf63e7d3.slice/crio-36e9c05bcbd4118c392a8feb3c695752c521bb0c9c9cdcc0d61e92d6df2afe28 WatchSource:0}: Error finding container 36e9c05bcbd4118c392a8feb3c695752c521bb0c9c9cdcc0d61e92d6df2afe28: Status 404 returned error can't find the container with id 36e9c05bcbd4118c392a8feb3c695752c521bb0c9c9cdcc0d61e92d6df2afe28 Jul 06 21:19:10.078638 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:10.078605 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7ccfc88547-nqxz9" event={"ID":"bfff6677-9f60-42a3-a03e-9dbfdf63e7d3","Type":"ContainerStarted","Data":"36e9c05bcbd4118c392a8feb3c695752c521bb0c9c9cdcc0d61e92d6df2afe28"} Jul 06 21:19:10.079760 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:10.079735 2577 generic.go:356] "Generic (PLEG): container finished" podID="79e9a705-3d7b-446f-85f1-7538ef885f41" containerID="221c00ee049d5cb6c32fcaba3c2f7c3764c717f118f9383b98c503ac95986ba4" exitCode=0 Jul 06 21:19:10.079858 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:10.079795 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" Jul 06 21:19:10.079858 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:10.079820 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" event={"ID":"79e9a705-3d7b-446f-85f1-7538ef885f41","Type":"ContainerDied","Data":"221c00ee049d5cb6c32fcaba3c2f7c3764c717f118f9383b98c503ac95986ba4"} Jul 06 21:19:10.079973 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:10.079872 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7ccfc88547-ftpdr" event={"ID":"79e9a705-3d7b-446f-85f1-7538ef885f41","Type":"ContainerDied","Data":"0a1877d1d3763107ed672250dff023f4a623368961c8d257c71df6a9c3cd0c43"} Jul 06 21:19:10.079973 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:10.079895 2577 scope.go:117] "RemoveContainer" containerID="221c00ee049d5cb6c32fcaba3c2f7c3764c717f118f9383b98c503ac95986ba4" Jul 06 21:19:10.088550 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:10.088532 2577 scope.go:117] "RemoveContainer" containerID="221c00ee049d5cb6c32fcaba3c2f7c3764c717f118f9383b98c503ac95986ba4" Jul 06 21:19:10.088821 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:19:10.088804 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"221c00ee049d5cb6c32fcaba3c2f7c3764c717f118f9383b98c503ac95986ba4\": container with ID starting with 221c00ee049d5cb6c32fcaba3c2f7c3764c717f118f9383b98c503ac95986ba4 not found: ID does not exist" containerID="221c00ee049d5cb6c32fcaba3c2f7c3764c717f118f9383b98c503ac95986ba4" Jul 06 21:19:10.088867 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:10.088828 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"221c00ee049d5cb6c32fcaba3c2f7c3764c717f118f9383b98c503ac95986ba4"} err="failed to get container status \"221c00ee049d5cb6c32fcaba3c2f7c3764c717f118f9383b98c503ac95986ba4\": rpc error: code = NotFound desc = could not find container \"221c00ee049d5cb6c32fcaba3c2f7c3764c717f118f9383b98c503ac95986ba4\": container with ID starting with 221c00ee049d5cb6c32fcaba3c2f7c3764c717f118f9383b98c503ac95986ba4 not found: ID does not exist" Jul 06 21:19:10.100873 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:10.100849 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7ccfc88547-ftpdr"] Jul 06 21:19:10.106100 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:10.106078 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-7ccfc88547-ftpdr"] Jul 06 21:19:10.489147 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:10.489072 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79e9a705-3d7b-446f-85f1-7538ef885f41" path="/var/lib/kubelet/pods/79e9a705-3d7b-446f-85f1-7538ef885f41/volumes" Jul 06 21:19:11.091324 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:11.091287 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7ccfc88547-nqxz9" event={"ID":"bfff6677-9f60-42a3-a03e-9dbfdf63e7d3","Type":"ContainerStarted","Data":"da1f95a12f1dddf8fad5454f22a9f33486f50b61ff3646432c6ed2f5ed076fcf"} Jul 06 21:19:11.091550 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:11.091422 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7ccfc88547-nqxz9" Jul 06 21:19:11.108804 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:11.108754 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7ccfc88547-nqxz9" podStartSLOduration=1.3774249539999999 podStartE2EDuration="2.108739415s" totalImagesPullingTime="731.314461ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:19:11.107152571 +0000 UTC m=+375.232134005" watchObservedRunningTime="2026-07-06 21:19:11.108739415 +0000 UTC m=+375.233720857" Jul 06 21:19:22.766951 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:22.766917 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:19:42.103258 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:42.103218 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7ccfc88547-nqxz9" Jul 06 21:19:43.047132 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:43.047087 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:19:43.135000 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:43.134966 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:19:43.264391 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:43.264355 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:19:43.366209 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:43.366171 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:19:43.856344 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:43.856317 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:19:43.922620 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:19:43.922587 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:20:23.871913 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:20:23.871821 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:20:58.454842 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:20:58.454800 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:20:58.557083 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:20:58.557038 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:09.062943 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:09.062906 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:09.164888 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:09.164844 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:09.225664 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:09.225628 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:14.913641 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:14.913576 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:20.111724 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:20.111689 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:20.214632 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:20.214597 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:31.574115 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:31.574068 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:31.674742 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:31.674712 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:31.842399 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:31.842305 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:31.992113 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:31.992074 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:32.055373 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:32.055329 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:32.151745 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:32.151667 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:32.253037 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:32.252995 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:32.340917 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:32.340880 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:32.521144 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:32.521065 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:21:32.975180 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:21:32.975146 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:48.784534 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:48.784498 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:51.668043 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.668013 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg"] Jul 06 21:22:51.668401 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.668336 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79e9a705-3d7b-446f-85f1-7538ef885f41" containerName="manager" Jul 06 21:22:51.668401 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.668347 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e9a705-3d7b-446f-85f1-7538ef885f41" containerName="manager" Jul 06 21:22:51.668401 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.668396 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="79e9a705-3d7b-446f-85f1-7538ef885f41" containerName="manager" Jul 06 21:22:51.671493 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.671449 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" Jul 06 21:22:51.673913 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.673876 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:51.674036 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.673910 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-dec69-serving-cert\"" Jul 06 21:22:51.674036 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.673923 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2qpmz\"" Jul 06 21:22:51.674036 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.673923 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-dec69-kube-rbac-proxy-sar-config\"" Jul 06 21:22:51.674775 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.674760 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 06 21:22:51.682515 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.682495 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg"] Jul 06 21:22:51.703949 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.703921 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab663b1-c3ce-4680-bccd-4fd23d550019-openshift-service-ca-bundle\") pod \"switch-graph-dec69-867bdb8f8c-zhrrg\" (UID: \"7ab663b1-c3ce-4680-bccd-4fd23d550019\") " pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" Jul 06 21:22:51.704099 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.703998 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ab663b1-c3ce-4680-bccd-4fd23d550019-proxy-tls\") pod \"switch-graph-dec69-867bdb8f8c-zhrrg\" (UID: \"7ab663b1-c3ce-4680-bccd-4fd23d550019\") " pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" Jul 06 21:22:51.794089 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.794047 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:51.804887 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.804851 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ab663b1-c3ce-4680-bccd-4fd23d550019-proxy-tls\") pod \"switch-graph-dec69-867bdb8f8c-zhrrg\" (UID: \"7ab663b1-c3ce-4680-bccd-4fd23d550019\") " pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" Jul 06 21:22:51.805059 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.804930 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab663b1-c3ce-4680-bccd-4fd23d550019-openshift-service-ca-bundle\") pod \"switch-graph-dec69-867bdb8f8c-zhrrg\" (UID: \"7ab663b1-c3ce-4680-bccd-4fd23d550019\") " pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" Jul 06 21:22:51.805599 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.805573 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab663b1-c3ce-4680-bccd-4fd23d550019-openshift-service-ca-bundle\") pod \"switch-graph-dec69-867bdb8f8c-zhrrg\" (UID: \"7ab663b1-c3ce-4680-bccd-4fd23d550019\") " pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" Jul 06 21:22:51.807255 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.807234 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ab663b1-c3ce-4680-bccd-4fd23d550019-proxy-tls\") pod \"switch-graph-dec69-867bdb8f8c-zhrrg\" (UID: \"7ab663b1-c3ce-4680-bccd-4fd23d550019\") " pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" Jul 06 21:22:51.983164 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:51.983049 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" Jul 06 21:22:52.114711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:52.114677 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg"] Jul 06 21:22:52.117686 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:22:52.117647 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ab663b1_c3ce_4680_bccd_4fd23d550019.slice/crio-cf547e9da6a02367d6743dec293c3472662fa5b28fc432a1ae4791b241c5c146 WatchSource:0}: Error finding container cf547e9da6a02367d6743dec293c3472662fa5b28fc432a1ae4791b241c5c146: Status 404 returned error can't find the container with id cf547e9da6a02367d6743dec293c3472662fa5b28fc432a1ae4791b241c5c146 Jul 06 21:22:53.009699 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:53.009660 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" event={"ID":"7ab663b1-c3ce-4680-bccd-4fd23d550019","Type":"ContainerStarted","Data":"cf547e9da6a02367d6743dec293c3472662fa5b28fc432a1ae4791b241c5c146"} Jul 06 21:22:55.020914 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:55.020878 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" event={"ID":"7ab663b1-c3ce-4680-bccd-4fd23d550019","Type":"ContainerStarted","Data":"63b7dd899bbd0c5a950e35f7bcb25146cfed43d3bc17451add53ae808d217ab2"} Jul 06 21:22:55.021290 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:55.021004 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" Jul 06 21:22:55.036695 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:55.036644 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" podStartSLOduration=1.665172907 podStartE2EDuration="4.036629963s" totalImagesPullingTime="2.371457056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:22:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:22:55.035405 +0000 UTC m=+599.160386423" watchObservedRunningTime="2026-07-06 21:22:55.036629963 +0000 UTC m=+599.161611418" Jul 06 21:22:56.316348 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:56.316318 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:22:56.316751 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:56.316422 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:22:56.319919 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:56.319901 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:22:56.319919 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:22:56.319915 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:23:01.031932 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:01.031901 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" Jul 06 21:23:01.785552 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:01.785519 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg"] Jul 06 21:23:01.785730 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:01.785690 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" podUID="7ab663b1-c3ce-4680-bccd-4fd23d550019" containerName="switch-graph-dec69" containerID="cri-o://63b7dd899bbd0c5a950e35f7bcb25146cfed43d3bc17451add53ae808d217ab2" gracePeriod=30 Jul 06 21:23:01.805852 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:01.805820 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:01.925354 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:01.925309 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:02.013384 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:02.013336 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:02.084558 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:02.084518 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:02.529228 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:02.529144 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:02.630147 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:02.630112 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:06.030421 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:06.030375 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" podUID="7ab663b1-c3ce-4680-bccd-4fd23d550019" containerName="switch-graph-dec69" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:23:11.030712 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:11.030669 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" podUID="7ab663b1-c3ce-4680-bccd-4fd23d550019" containerName="switch-graph-dec69" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:23:16.029804 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:16.029763 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" podUID="7ab663b1-c3ce-4680-bccd-4fd23d550019" containerName="switch-graph-dec69" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:23:16.030174 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:16.029886 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" Jul 06 21:23:21.029872 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:21.029778 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" podUID="7ab663b1-c3ce-4680-bccd-4fd23d550019" containerName="switch-graph-dec69" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:23:26.030515 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:26.030456 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" podUID="7ab663b1-c3ce-4680-bccd-4fd23d550019" containerName="switch-graph-dec69" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:23:31.029776 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:31.029734 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" podUID="7ab663b1-c3ce-4680-bccd-4fd23d550019" containerName="switch-graph-dec69" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:23:32.178947 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:32.178910 2577 generic.go:356] "Generic (PLEG): container finished" podID="7ab663b1-c3ce-4680-bccd-4fd23d550019" containerID="63b7dd899bbd0c5a950e35f7bcb25146cfed43d3bc17451add53ae808d217ab2" exitCode=0 Jul 06 21:23:32.178947 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:32.178950 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" event={"ID":"7ab663b1-c3ce-4680-bccd-4fd23d550019","Type":"ContainerDied","Data":"63b7dd899bbd0c5a950e35f7bcb25146cfed43d3bc17451add53ae808d217ab2"} Jul 06 21:23:32.432622 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:32.432562 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" Jul 06 21:23:32.540209 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:32.540176 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab663b1-c3ce-4680-bccd-4fd23d550019-openshift-service-ca-bundle\") pod \"7ab663b1-c3ce-4680-bccd-4fd23d550019\" (UID: \"7ab663b1-c3ce-4680-bccd-4fd23d550019\") " Jul 06 21:23:32.540384 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:32.540273 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ab663b1-c3ce-4680-bccd-4fd23d550019-proxy-tls\") pod \"7ab663b1-c3ce-4680-bccd-4fd23d550019\" (UID: \"7ab663b1-c3ce-4680-bccd-4fd23d550019\") " Jul 06 21:23:32.540615 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:32.540596 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ab663b1-c3ce-4680-bccd-4fd23d550019-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "7ab663b1-c3ce-4680-bccd-4fd23d550019" (UID: "7ab663b1-c3ce-4680-bccd-4fd23d550019"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:23:32.542370 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:32.542347 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ab663b1-c3ce-4680-bccd-4fd23d550019-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7ab663b1-c3ce-4680-bccd-4fd23d550019" (UID: "7ab663b1-c3ce-4680-bccd-4fd23d550019"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:23:32.641536 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:32.641490 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ab663b1-c3ce-4680-bccd-4fd23d550019-proxy-tls\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:23:32.641536 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:32.641536 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ab663b1-c3ce-4680-bccd-4fd23d550019-openshift-service-ca-bundle\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:23:33.184916 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:33.184874 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" event={"ID":"7ab663b1-c3ce-4680-bccd-4fd23d550019","Type":"ContainerDied","Data":"cf547e9da6a02367d6743dec293c3472662fa5b28fc432a1ae4791b241c5c146"} Jul 06 21:23:33.184916 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:33.184911 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg" Jul 06 21:23:33.187383 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:33.184969 2577 scope.go:117] "RemoveContainer" containerID="63b7dd899bbd0c5a950e35f7bcb25146cfed43d3bc17451add53ae808d217ab2" Jul 06 21:23:33.206040 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:33.206009 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg"] Jul 06 21:23:33.210334 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:33.210313 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-dec69-867bdb8f8c-zhrrg"] Jul 06 21:23:34.489653 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:34.489620 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ab663b1-c3ce-4680-bccd-4fd23d550019" path="/var/lib/kubelet/pods/7ab663b1-c3ce-4680-bccd-4fd23d550019/volumes" Jul 06 21:23:51.639640 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.639602 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch"] Jul 06 21:23:51.640030 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.639956 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ab663b1-c3ce-4680-bccd-4fd23d550019" containerName="switch-graph-dec69" Jul 06 21:23:51.640030 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.639971 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ab663b1-c3ce-4680-bccd-4fd23d550019" containerName="switch-graph-dec69" Jul 06 21:23:51.640111 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.640040 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ab663b1-c3ce-4680-bccd-4fd23d550019" containerName="switch-graph-dec69" Jul 06 21:23:51.642839 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.642820 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" Jul 06 21:23:51.645188 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.645166 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\"" Jul 06 21:23:51.645332 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.645223 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 06 21:23:51.645416 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.645366 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2qpmz\"" Jul 06 21:23:51.645506 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.645491 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\"" Jul 06 21:23:51.650286 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.650174 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch"] Jul 06 21:23:51.655518 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.655483 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:51.683192 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.683162 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b-openshift-service-ca-bundle\") pod \"model-chainer-6b7f89d56f-kddch\" (UID: \"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b\") " pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" Jul 06 21:23:51.683192 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.683195 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b-proxy-tls\") pod \"model-chainer-6b7f89d56f-kddch\" (UID: \"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b\") " pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" Jul 06 21:23:51.783789 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.783758 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b-openshift-service-ca-bundle\") pod \"model-chainer-6b7f89d56f-kddch\" (UID: \"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b\") " pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" Jul 06 21:23:51.783789 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.783791 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b-proxy-tls\") pod \"model-chainer-6b7f89d56f-kddch\" (UID: \"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b\") " pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" Jul 06 21:23:51.784419 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.784385 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b-openshift-service-ca-bundle\") pod \"model-chainer-6b7f89d56f-kddch\" (UID: \"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b\") " pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" Jul 06 21:23:51.786244 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.786222 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b-proxy-tls\") pod \"model-chainer-6b7f89d56f-kddch\" (UID: \"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b\") " pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" Jul 06 21:23:51.954533 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:51.954422 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" Jul 06 21:23:52.080481 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:52.080442 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch"] Jul 06 21:23:52.082903 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:23:52.082877 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd6be52e_b9e0_4a8b_bcca_1ce9db1ab02b.slice/crio-2e41f14d11b6083882e1b12b29a140ce163d750f3ba2fc9a4b7e6f99bc9fe857 WatchSource:0}: Error finding container 2e41f14d11b6083882e1b12b29a140ce163d750f3ba2fc9a4b7e6f99bc9fe857: Status 404 returned error can't find the container with id 2e41f14d11b6083882e1b12b29a140ce163d750f3ba2fc9a4b7e6f99bc9fe857 Jul 06 21:23:52.274568 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:52.274481 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" event={"ID":"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b","Type":"ContainerStarted","Data":"ed34c2789c50565a9660011ced91688e533687d9903bfbec8fae8be5602605f4"} Jul 06 21:23:52.274568 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:52.274555 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" event={"ID":"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b","Type":"ContainerStarted","Data":"2e41f14d11b6083882e1b12b29a140ce163d750f3ba2fc9a4b7e6f99bc9fe857"} Jul 06 21:23:52.274751 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:52.274623 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" Jul 06 21:23:52.292179 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:52.292133 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" podStartSLOduration=1.29211953 podStartE2EDuration="1.29211953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:23:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:23:52.290428637 +0000 UTC m=+656.415410061" watchObservedRunningTime="2026-07-06 21:23:52.29211953 +0000 UTC m=+656.417100952" Jul 06 21:23:52.951899 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:52.951829 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:58.285041 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:23:58.285010 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" Jul 06 21:24:01.735091 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:01.735055 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch"] Jul 06 21:24:01.735573 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:01.735306 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" podUID="cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" containerName="model-chainer" containerID="cri-o://ed34c2789c50565a9660011ced91688e533687d9903bfbec8fae8be5602605f4" gracePeriod=30 Jul 06 21:24:01.756796 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:01.756764 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:01.842092 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:01.842046 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:01.928848 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:01.928811 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:02.006706 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.006607 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:02.034768 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.034729 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5"] Jul 06 21:24:02.039313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.039285 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:24:02.041678 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.041654 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-0aa94-kube-rbac-proxy-sar-config\"" Jul 06 21:24:02.041819 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.041755 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-0aa94-serving-cert\"" Jul 06 21:24:02.046177 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.046150 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5"] Jul 06 21:24:02.063428 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.063395 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26567ec6-5d6b-4447-9735-30abcc8b0f6c-openshift-service-ca-bundle\") pod \"switch-graph-0aa94-57bc7d56f6-w2ts5\" (UID: \"26567ec6-5d6b-4447-9735-30abcc8b0f6c\") " pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:24:02.063664 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.063444 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/26567ec6-5d6b-4447-9735-30abcc8b0f6c-proxy-tls\") pod \"switch-graph-0aa94-57bc7d56f6-w2ts5\" (UID: \"26567ec6-5d6b-4447-9735-30abcc8b0f6c\") " pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:24:02.075065 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.075036 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:02.164092 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.164058 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26567ec6-5d6b-4447-9735-30abcc8b0f6c-openshift-service-ca-bundle\") pod \"switch-graph-0aa94-57bc7d56f6-w2ts5\" (UID: \"26567ec6-5d6b-4447-9735-30abcc8b0f6c\") " pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:24:02.164260 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.164100 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/26567ec6-5d6b-4447-9735-30abcc8b0f6c-proxy-tls\") pod \"switch-graph-0aa94-57bc7d56f6-w2ts5\" (UID: \"26567ec6-5d6b-4447-9735-30abcc8b0f6c\") " pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:24:02.164260 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:24:02.164233 2577 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-0aa94-serving-cert: secret "switch-graph-0aa94-serving-cert" not found Jul 06 21:24:02.164345 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:24:02.164302 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26567ec6-5d6b-4447-9735-30abcc8b0f6c-proxy-tls podName:26567ec6-5d6b-4447-9735-30abcc8b0f6c nodeName:}" failed. No retries permitted until 2026-07-06 21:24:02.664284246 +0000 UTC m=+666.789265652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/26567ec6-5d6b-4447-9735-30abcc8b0f6c-proxy-tls") pod "switch-graph-0aa94-57bc7d56f6-w2ts5" (UID: "26567ec6-5d6b-4447-9735-30abcc8b0f6c") : secret "switch-graph-0aa94-serving-cert" not found Jul 06 21:24:02.164817 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.164797 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26567ec6-5d6b-4447-9735-30abcc8b0f6c-openshift-service-ca-bundle\") pod \"switch-graph-0aa94-57bc7d56f6-w2ts5\" (UID: \"26567ec6-5d6b-4447-9735-30abcc8b0f6c\") " pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:24:02.233892 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.233856 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:02.334675 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.334638 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:02.668342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.668256 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/26567ec6-5d6b-4447-9735-30abcc8b0f6c-proxy-tls\") pod \"switch-graph-0aa94-57bc7d56f6-w2ts5\" (UID: \"26567ec6-5d6b-4447-9735-30abcc8b0f6c\") " pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:24:02.670709 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.670684 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/26567ec6-5d6b-4447-9735-30abcc8b0f6c-proxy-tls\") pod \"switch-graph-0aa94-57bc7d56f6-w2ts5\" (UID: \"26567ec6-5d6b-4447-9735-30abcc8b0f6c\") " pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:24:02.950417 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:02.950338 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:24:03.083001 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:03.082963 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5"] Jul 06 21:24:03.086116 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:24:03.086079 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26567ec6_5d6b_4447_9735_30abcc8b0f6c.slice/crio-d53c82e01b4157ca67dde0dec46be84a20e885925f94db1afa6dbb92bcacb2f6 WatchSource:0}: Error finding container d53c82e01b4157ca67dde0dec46be84a20e885925f94db1afa6dbb92bcacb2f6: Status 404 returned error can't find the container with id d53c82e01b4157ca67dde0dec46be84a20e885925f94db1afa6dbb92bcacb2f6 Jul 06 21:24:03.282484 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:03.282370 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" podUID="cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:24:03.321610 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:03.321574 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" event={"ID":"26567ec6-5d6b-4447-9735-30abcc8b0f6c","Type":"ContainerStarted","Data":"ae6f7d9946c31c43f81917597877960828149b9765a8058cf917c1e6e309911b"} Jul 06 21:24:03.321777 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:03.321625 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" event={"ID":"26567ec6-5d6b-4447-9735-30abcc8b0f6c","Type":"ContainerStarted","Data":"d53c82e01b4157ca67dde0dec46be84a20e885925f94db1afa6dbb92bcacb2f6"} Jul 06 21:24:03.321777 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:03.321728 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:24:03.336889 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:03.336841 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" podStartSLOduration=1.336826829 podStartE2EDuration="1.336826829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:24:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:24:03.335900236 +0000 UTC m=+667.460881657" watchObservedRunningTime="2026-07-06 21:24:03.336826829 +0000 UTC m=+667.461808272" Jul 06 21:24:08.282826 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:08.282781 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" podUID="cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:24:09.330918 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:09.330886 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:24:10.248676 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:10.248639 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:10.315389 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:10.315357 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:13.282920 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:13.282873 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" podUID="cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:24:13.283352 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:13.283037 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" Jul 06 21:24:18.282567 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:18.282514 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" podUID="cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:24:23.282983 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:23.282938 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" podUID="cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:24:27.357962 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:27.357924 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:27.459262 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:27.459231 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:28.283101 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:28.283057 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" podUID="cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:24:31.885750 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:31.885726 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" Jul 06 21:24:32.016955 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.016859 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b-proxy-tls\") pod \"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b\" (UID: \"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b\") " Jul 06 21:24:32.016955 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.016913 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b-openshift-service-ca-bundle\") pod \"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b\" (UID: \"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b\") " Jul 06 21:24:32.017300 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.017278 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" (UID: "cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:24:32.018936 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.018905 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" (UID: "cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:24:32.117815 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.117781 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b-proxy-tls\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:24:32.117975 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.117810 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b-openshift-service-ca-bundle\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:24:32.450518 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.450484 2577 generic.go:356] "Generic (PLEG): container finished" podID="cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" containerID="ed34c2789c50565a9660011ced91688e533687d9903bfbec8fae8be5602605f4" exitCode=137 Jul 06 21:24:32.450711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.450530 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" event={"ID":"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b","Type":"ContainerDied","Data":"ed34c2789c50565a9660011ced91688e533687d9903bfbec8fae8be5602605f4"} Jul 06 21:24:32.450711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.450535 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" Jul 06 21:24:32.450711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.450580 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch" event={"ID":"cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b","Type":"ContainerDied","Data":"2e41f14d11b6083882e1b12b29a140ce163d750f3ba2fc9a4b7e6f99bc9fe857"} Jul 06 21:24:32.450711 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.450622 2577 scope.go:117] "RemoveContainer" containerID="ed34c2789c50565a9660011ced91688e533687d9903bfbec8fae8be5602605f4" Jul 06 21:24:32.459068 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.459043 2577 scope.go:117] "RemoveContainer" containerID="ed34c2789c50565a9660011ced91688e533687d9903bfbec8fae8be5602605f4" Jul 06 21:24:32.459303 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:24:32.459284 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed34c2789c50565a9660011ced91688e533687d9903bfbec8fae8be5602605f4\": container with ID starting with ed34c2789c50565a9660011ced91688e533687d9903bfbec8fae8be5602605f4 not found: ID does not exist" containerID="ed34c2789c50565a9660011ced91688e533687d9903bfbec8fae8be5602605f4" Jul 06 21:24:32.459356 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.459315 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed34c2789c50565a9660011ced91688e533687d9903bfbec8fae8be5602605f4"} err="failed to get container status \"ed34c2789c50565a9660011ced91688e533687d9903bfbec8fae8be5602605f4\": rpc error: code = NotFound desc = could not find container \"ed34c2789c50565a9660011ced91688e533687d9903bfbec8fae8be5602605f4\": container with ID starting with ed34c2789c50565a9660011ced91688e533687d9903bfbec8fae8be5602605f4 not found: ID does not exist" Jul 06 21:24:32.471432 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.471409 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch"] Jul 06 21:24:32.475408 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.475386 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6b7f89d56f-kddch"] Jul 06 21:24:32.489049 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:32.489028 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" path="/var/lib/kubelet/pods/cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b/volumes" Jul 06 21:24:54.164043 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:24:54.164002 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:01.911277 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:01.911240 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz"] Jul 06 21:25:01.911750 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:01.911665 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" containerName="model-chainer" Jul 06 21:25:01.911750 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:01.911680 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" containerName="model-chainer" Jul 06 21:25:01.911750 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:01.911750 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd6be52e-b9e0-4a8b-bcca-1ce9db1ab02b" containerName="model-chainer" Jul 06 21:25:01.915037 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:01.915015 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" Jul 06 21:25:01.917517 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:01.917482 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-22b2a-serving-cert\"" Jul 06 21:25:01.917642 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:01.917544 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-22b2a-kube-rbac-proxy-sar-config\"" Jul 06 21:25:01.925777 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:01.925749 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af66df17-18dc-4522-9918-14e92e52f1b6-openshift-service-ca-bundle\") pod \"sequence-graph-22b2a-5fc98d8d55-mzbmz\" (UID: \"af66df17-18dc-4522-9918-14e92e52f1b6\") " pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" Jul 06 21:25:01.925896 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:01.925793 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af66df17-18dc-4522-9918-14e92e52f1b6-proxy-tls\") pod \"sequence-graph-22b2a-5fc98d8d55-mzbmz\" (UID: \"af66df17-18dc-4522-9918-14e92e52f1b6\") " pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" Jul 06 21:25:01.926337 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:01.926316 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz"] Jul 06 21:25:01.929666 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:01.929639 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:02.026321 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:02.026283 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af66df17-18dc-4522-9918-14e92e52f1b6-openshift-service-ca-bundle\") pod \"sequence-graph-22b2a-5fc98d8d55-mzbmz\" (UID: \"af66df17-18dc-4522-9918-14e92e52f1b6\") " pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" Jul 06 21:25:02.026571 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:02.026332 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af66df17-18dc-4522-9918-14e92e52f1b6-proxy-tls\") pod \"sequence-graph-22b2a-5fc98d8d55-mzbmz\" (UID: \"af66df17-18dc-4522-9918-14e92e52f1b6\") " pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" Jul 06 21:25:02.026995 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:02.026970 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af66df17-18dc-4522-9918-14e92e52f1b6-openshift-service-ca-bundle\") pod \"sequence-graph-22b2a-5fc98d8d55-mzbmz\" (UID: \"af66df17-18dc-4522-9918-14e92e52f1b6\") " pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" Jul 06 21:25:02.029028 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:02.028985 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:02.029192 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:02.029173 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af66df17-18dc-4522-9918-14e92e52f1b6-proxy-tls\") pod \"sequence-graph-22b2a-5fc98d8d55-mzbmz\" (UID: \"af66df17-18dc-4522-9918-14e92e52f1b6\") " pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" Jul 06 21:25:02.226173 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:02.226079 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" Jul 06 21:25:02.357228 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:02.357200 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz"] Jul 06 21:25:02.359534 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:25:02.359506 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf66df17_18dc_4522_9918_14e92e52f1b6.slice/crio-3b828b8e857fc41e91043fede4de257421f9c66df3350ffe2def8f3c791c6f75 WatchSource:0}: Error finding container 3b828b8e857fc41e91043fede4de257421f9c66df3350ffe2def8f3c791c6f75: Status 404 returned error can't find the container with id 3b828b8e857fc41e91043fede4de257421f9c66df3350ffe2def8f3c791c6f75 Jul 06 21:25:02.578065 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:02.578031 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" event={"ID":"af66df17-18dc-4522-9918-14e92e52f1b6","Type":"ContainerStarted","Data":"212e9f952a71bcdac3d4772d585d45f561ca5c021ebf1e83d1f69a788c4433ea"} Jul 06 21:25:02.578253 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:02.578079 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" event={"ID":"af66df17-18dc-4522-9918-14e92e52f1b6","Type":"ContainerStarted","Data":"3b828b8e857fc41e91043fede4de257421f9c66df3350ffe2def8f3c791c6f75"} Jul 06 21:25:02.578253 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:02.578166 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" Jul 06 21:25:02.595198 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:02.595152 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" podStartSLOduration=1.5951373439999998 podStartE2EDuration="1.595137344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:25:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:25:02.594338292 +0000 UTC m=+726.719319714" watchObservedRunningTime="2026-07-06 21:25:02.595137344 +0000 UTC m=+726.720118766" Jul 06 21:25:08.587922 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:08.587889 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" Jul 06 21:25:21.424285 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:21.424243 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:32.298066 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:25:32.298032 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:04.081637 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:27:04.081607 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:56.342086 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:27:56.342052 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:27:56.343019 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:27:56.342998 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:27:56.345418 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:27:56.345396 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:27:56.346310 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:27:56.346294 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:29:12.301956 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:29:12.301916 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:30:06.355724 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:30:06.355685 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:30:06.456417 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:30:06.456375 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:16.677904 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:16.677870 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5"] Jul 06 21:32:16.678506 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:16.678095 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" podUID="26567ec6-5d6b-4447-9735-30abcc8b0f6c" containerName="switch-graph-0aa94" containerID="cri-o://ae6f7d9946c31c43f81917597877960828149b9765a8058cf917c1e6e309911b" gracePeriod=30 Jul 06 21:32:16.781111 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:16.781078 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:16.876591 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:16.876558 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:16.963402 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:16.963323 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:17.047408 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:17.047376 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:17.144453 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:17.144400 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:17.243994 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:17.243900 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:19.329141 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:19.329056 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" podUID="26567ec6-5d6b-4447-9735-30abcc8b0f6c" containerName="switch-graph-0aa94" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:32:24.329503 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:24.329447 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" podUID="26567ec6-5d6b-4447-9735-30abcc8b0f6c" containerName="switch-graph-0aa94" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:32:29.329349 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:29.329308 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" podUID="26567ec6-5d6b-4447-9735-30abcc8b0f6c" containerName="switch-graph-0aa94" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:32:29.329820 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:29.329428 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:32:34.329064 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:34.329025 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" podUID="26567ec6-5d6b-4447-9735-30abcc8b0f6c" containerName="switch-graph-0aa94" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:32:39.328887 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:39.328832 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" podUID="26567ec6-5d6b-4447-9735-30abcc8b0f6c" containerName="switch-graph-0aa94" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:32:44.329763 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:44.329720 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" podUID="26567ec6-5d6b-4447-9735-30abcc8b0f6c" containerName="switch-graph-0aa94" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:32:46.831063 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:46.831040 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:32:46.885651 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:46.885619 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26567ec6-5d6b-4447-9735-30abcc8b0f6c-openshift-service-ca-bundle\") pod \"26567ec6-5d6b-4447-9735-30abcc8b0f6c\" (UID: \"26567ec6-5d6b-4447-9735-30abcc8b0f6c\") " Jul 06 21:32:46.885829 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:46.885728 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/26567ec6-5d6b-4447-9735-30abcc8b0f6c-proxy-tls\") pod \"26567ec6-5d6b-4447-9735-30abcc8b0f6c\" (UID: \"26567ec6-5d6b-4447-9735-30abcc8b0f6c\") " Jul 06 21:32:46.886000 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:46.885978 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26567ec6-5d6b-4447-9735-30abcc8b0f6c-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "26567ec6-5d6b-4447-9735-30abcc8b0f6c" (UID: "26567ec6-5d6b-4447-9735-30abcc8b0f6c"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:32:46.887831 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:46.887810 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26567ec6-5d6b-4447-9735-30abcc8b0f6c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "26567ec6-5d6b-4447-9735-30abcc8b0f6c" (UID: "26567ec6-5d6b-4447-9735-30abcc8b0f6c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:32:46.986538 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:46.986425 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26567ec6-5d6b-4447-9735-30abcc8b0f6c-openshift-service-ca-bundle\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:32:46.986538 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:46.986457 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/26567ec6-5d6b-4447-9735-30abcc8b0f6c-proxy-tls\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:32:47.563239 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:47.563201 2577 generic.go:356] "Generic (PLEG): container finished" podID="26567ec6-5d6b-4447-9735-30abcc8b0f6c" containerID="ae6f7d9946c31c43f81917597877960828149b9765a8058cf917c1e6e309911b" exitCode=0 Jul 06 21:32:47.563437 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:47.563251 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" event={"ID":"26567ec6-5d6b-4447-9735-30abcc8b0f6c","Type":"ContainerDied","Data":"ae6f7d9946c31c43f81917597877960828149b9765a8058cf917c1e6e309911b"} Jul 06 21:32:47.563437 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:47.563252 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" Jul 06 21:32:47.563437 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:47.563297 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5" event={"ID":"26567ec6-5d6b-4447-9735-30abcc8b0f6c","Type":"ContainerDied","Data":"d53c82e01b4157ca67dde0dec46be84a20e885925f94db1afa6dbb92bcacb2f6"} Jul 06 21:32:47.563437 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:47.563322 2577 scope.go:117] "RemoveContainer" containerID="ae6f7d9946c31c43f81917597877960828149b9765a8058cf917c1e6e309911b" Jul 06 21:32:47.571773 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:47.571754 2577 scope.go:117] "RemoveContainer" containerID="ae6f7d9946c31c43f81917597877960828149b9765a8058cf917c1e6e309911b" Jul 06 21:32:47.572060 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:32:47.572033 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae6f7d9946c31c43f81917597877960828149b9765a8058cf917c1e6e309911b\": container with ID starting with ae6f7d9946c31c43f81917597877960828149b9765a8058cf917c1e6e309911b not found: ID does not exist" containerID="ae6f7d9946c31c43f81917597877960828149b9765a8058cf917c1e6e309911b" Jul 06 21:32:47.572143 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:47.572071 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae6f7d9946c31c43f81917597877960828149b9765a8058cf917c1e6e309911b"} err="failed to get container status \"ae6f7d9946c31c43f81917597877960828149b9765a8058cf917c1e6e309911b\": rpc error: code = NotFound desc = could not find container \"ae6f7d9946c31c43f81917597877960828149b9765a8058cf917c1e6e309911b\": container with ID starting with ae6f7d9946c31c43f81917597877960828149b9765a8058cf917c1e6e309911b not found: ID does not exist" Jul 06 21:32:47.584250 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:47.584220 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5"] Jul 06 21:32:47.588024 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:47.588001 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0aa94-57bc7d56f6-w2ts5"] Jul 06 21:32:48.490507 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:48.490452 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26567ec6-5d6b-4447-9735-30abcc8b0f6c" path="/var/lib/kubelet/pods/26567ec6-5d6b-4447-9735-30abcc8b0f6c/volumes" Jul 06 21:32:56.369026 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:56.368992 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:32:56.371253 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:56.371234 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:32:56.372600 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:56.372582 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:32:56.375067 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:32:56.375048 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:33:16.653206 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.653173 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz"] Jul 06 21:33:16.653624 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.653368 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" podUID="af66df17-18dc-4522-9918-14e92e52f1b6" containerName="sequence-graph-22b2a" containerID="cri-o://212e9f952a71bcdac3d4772d585d45f561ca5c021ebf1e83d1f69a788c4433ea" gracePeriod=30 Jul 06 21:33:16.673682 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.673653 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:16.754369 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.754341 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:16.840737 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.840700 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:16.903285 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.903203 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58"] Jul 06 21:33:16.903561 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.903547 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26567ec6-5d6b-4447-9735-30abcc8b0f6c" containerName="switch-graph-0aa94" Jul 06 21:33:16.903610 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.903562 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="26567ec6-5d6b-4447-9735-30abcc8b0f6c" containerName="switch-graph-0aa94" Jul 06 21:33:16.903644 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.903620 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="26567ec6-5d6b-4447-9735-30abcc8b0f6c" containerName="switch-graph-0aa94" Jul 06 21:33:16.910812 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.910778 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" Jul 06 21:33:16.913854 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.913825 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-69e1e-kube-rbac-proxy-sar-config\"" Jul 06 21:33:16.913984 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.913851 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-69e1e-serving-cert\"" Jul 06 21:33:16.917729 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.917709 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58"] Jul 06 21:33:16.925957 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:16.925934 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:17.020999 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.020957 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:17.031174 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.031141 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c140bb7-79da-4e58-9da9-c5aef09c2fcd-openshift-service-ca-bundle\") pod \"ensemble-graph-69e1e-5cc86cdc8b-bnz58\" (UID: \"2c140bb7-79da-4e58-9da9-c5aef09c2fcd\") " pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" Jul 06 21:33:17.031174 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.031177 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2c140bb7-79da-4e58-9da9-c5aef09c2fcd-proxy-tls\") pod \"ensemble-graph-69e1e-5cc86cdc8b-bnz58\" (UID: \"2c140bb7-79da-4e58-9da9-c5aef09c2fcd\") " pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" Jul 06 21:33:17.118354 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.118323 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:17.132537 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.132508 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c140bb7-79da-4e58-9da9-c5aef09c2fcd-openshift-service-ca-bundle\") pod \"ensemble-graph-69e1e-5cc86cdc8b-bnz58\" (UID: \"2c140bb7-79da-4e58-9da9-c5aef09c2fcd\") " pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" Jul 06 21:33:17.132683 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.132542 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2c140bb7-79da-4e58-9da9-c5aef09c2fcd-proxy-tls\") pod \"ensemble-graph-69e1e-5cc86cdc8b-bnz58\" (UID: \"2c140bb7-79da-4e58-9da9-c5aef09c2fcd\") " pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" Jul 06 21:33:17.133148 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.133129 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c140bb7-79da-4e58-9da9-c5aef09c2fcd-openshift-service-ca-bundle\") pod \"ensemble-graph-69e1e-5cc86cdc8b-bnz58\" (UID: \"2c140bb7-79da-4e58-9da9-c5aef09c2fcd\") " pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" Jul 06 21:33:17.134984 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.134960 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2c140bb7-79da-4e58-9da9-c5aef09c2fcd-proxy-tls\") pod \"ensemble-graph-69e1e-5cc86cdc8b-bnz58\" (UID: \"2c140bb7-79da-4e58-9da9-c5aef09c2fcd\") " pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" Jul 06 21:33:17.219023 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.218928 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:17.222935 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.222916 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" Jul 06 21:33:17.355451 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.355425 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58"] Jul 06 21:33:17.358005 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:33:17.357970 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c140bb7_79da_4e58_9da9_c5aef09c2fcd.slice/crio-1e4189d7394d569da2c253e58b5e5efcdb15ba475b67e9284c5358457a1b5946 WatchSource:0}: Error finding container 1e4189d7394d569da2c253e58b5e5efcdb15ba475b67e9284c5358457a1b5946: Status 404 returned error can't find the container with id 1e4189d7394d569da2c253e58b5e5efcdb15ba475b67e9284c5358457a1b5946 Jul 06 21:33:17.689733 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.689701 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" event={"ID":"2c140bb7-79da-4e58-9da9-c5aef09c2fcd","Type":"ContainerStarted","Data":"ead01753f4f8a9f70c0671c75fc711c19ba8ae8e910140ed9403b796238c6681"} Jul 06 21:33:17.690137 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.689752 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" event={"ID":"2c140bb7-79da-4e58-9da9-c5aef09c2fcd","Type":"ContainerStarted","Data":"1e4189d7394d569da2c253e58b5e5efcdb15ba475b67e9284c5358457a1b5946"} Jul 06 21:33:17.690137 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.689798 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" Jul 06 21:33:17.707811 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:17.707761 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" podStartSLOduration=1.707747895 podStartE2EDuration="1.707747895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:33:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:33:17.70578714 +0000 UTC m=+1221.830768575" watchObservedRunningTime="2026-07-06 21:33:17.707747895 +0000 UTC m=+1221.832729318" Jul 06 21:33:18.586601 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:18.586552 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" podUID="af66df17-18dc-4522-9918-14e92e52f1b6" containerName="sequence-graph-22b2a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:33:23.586104 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:23.586058 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" podUID="af66df17-18dc-4522-9918-14e92e52f1b6" containerName="sequence-graph-22b2a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:33:23.699413 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:23.699378 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" Jul 06 21:33:26.957913 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:26.957833 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58"] Jul 06 21:33:26.958307 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:26.958006 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" podUID="2c140bb7-79da-4e58-9da9-c5aef09c2fcd" containerName="ensemble-graph-69e1e" containerID="cri-o://ead01753f4f8a9f70c0671c75fc711c19ba8ae8e910140ed9403b796238c6681" gracePeriod=30 Jul 06 21:33:26.991583 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:26.991549 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:27.108816 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:27.108780 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:27.179017 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:27.178979 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:27.279789 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:27.279715 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:27.345733 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:27.345700 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:27.446563 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:27.446531 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:27.563751 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:27.563667 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:28.586775 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:28.586734 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" podUID="af66df17-18dc-4522-9918-14e92e52f1b6" containerName="sequence-graph-22b2a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:33:28.587197 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:28.586833 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" Jul 06 21:33:28.697920 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:28.697880 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" podUID="2c140bb7-79da-4e58-9da9-c5aef09c2fcd" containerName="ensemble-graph-69e1e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:33:33.586176 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:33.586138 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" podUID="af66df17-18dc-4522-9918-14e92e52f1b6" containerName="sequence-graph-22b2a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:33:33.698360 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:33.698320 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" podUID="2c140bb7-79da-4e58-9da9-c5aef09c2fcd" containerName="ensemble-graph-69e1e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:33:38.585898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:38.585856 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" podUID="af66df17-18dc-4522-9918-14e92e52f1b6" containerName="sequence-graph-22b2a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:33:38.697916 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:38.697869 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" podUID="2c140bb7-79da-4e58-9da9-c5aef09c2fcd" containerName="ensemble-graph-69e1e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:33:38.698078 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:38.697990 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" Jul 06 21:33:39.640390 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:39.640355 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:43.586804 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:43.586756 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" podUID="af66df17-18dc-4522-9918-14e92e52f1b6" containerName="sequence-graph-22b2a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:33:43.698050 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:43.698004 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" podUID="2c140bb7-79da-4e58-9da9-c5aef09c2fcd" containerName="ensemble-graph-69e1e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:33:46.801837 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.801809 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" Jul 06 21:33:46.810454 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.810425 2577 generic.go:356] "Generic (PLEG): container finished" podID="af66df17-18dc-4522-9918-14e92e52f1b6" containerID="212e9f952a71bcdac3d4772d585d45f561ca5c021ebf1e83d1f69a788c4433ea" exitCode=0 Jul 06 21:33:46.810610 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.810476 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" Jul 06 21:33:46.810610 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.810504 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" event={"ID":"af66df17-18dc-4522-9918-14e92e52f1b6","Type":"ContainerDied","Data":"212e9f952a71bcdac3d4772d585d45f561ca5c021ebf1e83d1f69a788c4433ea"} Jul 06 21:33:46.810610 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.810560 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz" event={"ID":"af66df17-18dc-4522-9918-14e92e52f1b6","Type":"ContainerDied","Data":"3b828b8e857fc41e91043fede4de257421f9c66df3350ffe2def8f3c791c6f75"} Jul 06 21:33:46.810610 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.810585 2577 scope.go:117] "RemoveContainer" containerID="212e9f952a71bcdac3d4772d585d45f561ca5c021ebf1e83d1f69a788c4433ea" Jul 06 21:33:46.818477 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.818444 2577 scope.go:117] "RemoveContainer" containerID="212e9f952a71bcdac3d4772d585d45f561ca5c021ebf1e83d1f69a788c4433ea" Jul 06 21:33:46.818788 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:33:46.818757 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"212e9f952a71bcdac3d4772d585d45f561ca5c021ebf1e83d1f69a788c4433ea\": container with ID starting with 212e9f952a71bcdac3d4772d585d45f561ca5c021ebf1e83d1f69a788c4433ea not found: ID does not exist" containerID="212e9f952a71bcdac3d4772d585d45f561ca5c021ebf1e83d1f69a788c4433ea" Jul 06 21:33:46.818838 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.818808 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"212e9f952a71bcdac3d4772d585d45f561ca5c021ebf1e83d1f69a788c4433ea"} err="failed to get container status \"212e9f952a71bcdac3d4772d585d45f561ca5c021ebf1e83d1f69a788c4433ea\": rpc error: code = NotFound desc = could not find container \"212e9f952a71bcdac3d4772d585d45f561ca5c021ebf1e83d1f69a788c4433ea\": container with ID starting with 212e9f952a71bcdac3d4772d585d45f561ca5c021ebf1e83d1f69a788c4433ea not found: ID does not exist" Jul 06 21:33:46.864758 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.864727 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af66df17-18dc-4522-9918-14e92e52f1b6-openshift-service-ca-bundle\") pod \"af66df17-18dc-4522-9918-14e92e52f1b6\" (UID: \"af66df17-18dc-4522-9918-14e92e52f1b6\") " Jul 06 21:33:46.864932 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.864776 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af66df17-18dc-4522-9918-14e92e52f1b6-proxy-tls\") pod \"af66df17-18dc-4522-9918-14e92e52f1b6\" (UID: \"af66df17-18dc-4522-9918-14e92e52f1b6\") " Jul 06 21:33:46.865116 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.865095 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af66df17-18dc-4522-9918-14e92e52f1b6-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "af66df17-18dc-4522-9918-14e92e52f1b6" (UID: "af66df17-18dc-4522-9918-14e92e52f1b6"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:33:46.866880 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.866844 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af66df17-18dc-4522-9918-14e92e52f1b6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "af66df17-18dc-4522-9918-14e92e52f1b6" (UID: "af66df17-18dc-4522-9918-14e92e52f1b6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:33:46.965689 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.965605 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af66df17-18dc-4522-9918-14e92e52f1b6-openshift-service-ca-bundle\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:33:46.965689 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:46.965635 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/af66df17-18dc-4522-9918-14e92e52f1b6-proxy-tls\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:33:47.132257 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:47.132229 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz"] Jul 06 21:33:47.136521 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:47.136492 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-22b2a-5fc98d8d55-mzbmz"] Jul 06 21:33:48.489437 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:48.489337 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af66df17-18dc-4522-9918-14e92e52f1b6" path="/var/lib/kubelet/pods/af66df17-18dc-4522-9918-14e92e52f1b6/volumes" Jul 06 21:33:48.697920 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:48.697880 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" podUID="2c140bb7-79da-4e58-9da9-c5aef09c2fcd" containerName="ensemble-graph-69e1e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:33:53.698115 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:53.698076 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" podUID="2c140bb7-79da-4e58-9da9-c5aef09c2fcd" containerName="ensemble-graph-69e1e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:33:57.109679 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.109655 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" Jul 06 21:33:57.145660 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.145629 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c140bb7-79da-4e58-9da9-c5aef09c2fcd-openshift-service-ca-bundle\") pod \"2c140bb7-79da-4e58-9da9-c5aef09c2fcd\" (UID: \"2c140bb7-79da-4e58-9da9-c5aef09c2fcd\") " Jul 06 21:33:57.145806 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.145681 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2c140bb7-79da-4e58-9da9-c5aef09c2fcd-proxy-tls\") pod \"2c140bb7-79da-4e58-9da9-c5aef09c2fcd\" (UID: \"2c140bb7-79da-4e58-9da9-c5aef09c2fcd\") " Jul 06 21:33:57.145966 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.145941 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c140bb7-79da-4e58-9da9-c5aef09c2fcd-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "2c140bb7-79da-4e58-9da9-c5aef09c2fcd" (UID: "2c140bb7-79da-4e58-9da9-c5aef09c2fcd"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:33:57.147629 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.147598 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c140bb7-79da-4e58-9da9-c5aef09c2fcd-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2c140bb7-79da-4e58-9da9-c5aef09c2fcd" (UID: "2c140bb7-79da-4e58-9da9-c5aef09c2fcd"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:33:57.246294 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.246201 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c140bb7-79da-4e58-9da9-c5aef09c2fcd-openshift-service-ca-bundle\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:33:57.246294 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.246235 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2c140bb7-79da-4e58-9da9-c5aef09c2fcd-proxy-tls\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:33:57.858579 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.858545 2577 generic.go:356] "Generic (PLEG): container finished" podID="2c140bb7-79da-4e58-9da9-c5aef09c2fcd" containerID="ead01753f4f8a9f70c0671c75fc711c19ba8ae8e910140ed9403b796238c6681" exitCode=0 Jul 06 21:33:57.858757 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.858585 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" Jul 06 21:33:57.858757 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.858602 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" event={"ID":"2c140bb7-79da-4e58-9da9-c5aef09c2fcd","Type":"ContainerDied","Data":"ead01753f4f8a9f70c0671c75fc711c19ba8ae8e910140ed9403b796238c6681"} Jul 06 21:33:57.858757 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.858659 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58" event={"ID":"2c140bb7-79da-4e58-9da9-c5aef09c2fcd","Type":"ContainerDied","Data":"1e4189d7394d569da2c253e58b5e5efcdb15ba475b67e9284c5358457a1b5946"} Jul 06 21:33:57.858757 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.858684 2577 scope.go:117] "RemoveContainer" containerID="ead01753f4f8a9f70c0671c75fc711c19ba8ae8e910140ed9403b796238c6681" Jul 06 21:33:57.867074 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.867054 2577 scope.go:117] "RemoveContainer" containerID="ead01753f4f8a9f70c0671c75fc711c19ba8ae8e910140ed9403b796238c6681" Jul 06 21:33:57.867325 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:33:57.867303 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ead01753f4f8a9f70c0671c75fc711c19ba8ae8e910140ed9403b796238c6681\": container with ID starting with ead01753f4f8a9f70c0671c75fc711c19ba8ae8e910140ed9403b796238c6681 not found: ID does not exist" containerID="ead01753f4f8a9f70c0671c75fc711c19ba8ae8e910140ed9403b796238c6681" Jul 06 21:33:57.867379 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.867338 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead01753f4f8a9f70c0671c75fc711c19ba8ae8e910140ed9403b796238c6681"} err="failed to get container status \"ead01753f4f8a9f70c0671c75fc711c19ba8ae8e910140ed9403b796238c6681\": rpc error: code = NotFound desc = could not find container \"ead01753f4f8a9f70c0671c75fc711c19ba8ae8e910140ed9403b796238c6681\": container with ID starting with ead01753f4f8a9f70c0671c75fc711c19ba8ae8e910140ed9403b796238c6681 not found: ID does not exist" Jul 06 21:33:57.882534 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.882512 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58"] Jul 06 21:33:57.885963 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:57.885943 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-69e1e-5cc86cdc8b-bnz58"] Jul 06 21:33:58.490599 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:33:58.490568 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c140bb7-79da-4e58-9da9-c5aef09c2fcd" path="/var/lib/kubelet/pods/2c140bb7-79da-4e58-9da9-c5aef09c2fcd/volumes" Jul 06 21:34:26.801222 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.801189 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq"] Jul 06 21:34:26.801658 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.801528 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af66df17-18dc-4522-9918-14e92e52f1b6" containerName="sequence-graph-22b2a" Jul 06 21:34:26.801658 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.801541 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="af66df17-18dc-4522-9918-14e92e52f1b6" containerName="sequence-graph-22b2a" Jul 06 21:34:26.801658 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.801567 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c140bb7-79da-4e58-9da9-c5aef09c2fcd" containerName="ensemble-graph-69e1e" Jul 06 21:34:26.801658 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.801574 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c140bb7-79da-4e58-9da9-c5aef09c2fcd" containerName="ensemble-graph-69e1e" Jul 06 21:34:26.801658 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.801627 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="af66df17-18dc-4522-9918-14e92e52f1b6" containerName="sequence-graph-22b2a" Jul 06 21:34:26.801658 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.801637 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c140bb7-79da-4e58-9da9-c5aef09c2fcd" containerName="ensemble-graph-69e1e" Jul 06 21:34:26.804412 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.804391 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:34:26.806817 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.806782 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-599ee-kube-rbac-proxy-sar-config\"" Jul 06 21:34:26.806817 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.806792 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2qpmz\"" Jul 06 21:34:26.807012 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.806899 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-599ee-serving-cert\"" Jul 06 21:34:26.807789 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.807770 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 06 21:34:26.812333 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.812308 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq"] Jul 06 21:34:26.814525 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.814496 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:26.889091 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.889009 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-proxy-tls\") pod \"sequence-graph-599ee-6fb48cd747-x5wbq\" (UID: \"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8\") " pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:34:26.889091 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.889047 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-openshift-service-ca-bundle\") pod \"sequence-graph-599ee-6fb48cd747-x5wbq\" (UID: \"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8\") " pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:34:26.990317 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.990281 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-proxy-tls\") pod \"sequence-graph-599ee-6fb48cd747-x5wbq\" (UID: \"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8\") " pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:34:26.990317 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.990317 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-openshift-service-ca-bundle\") pod \"sequence-graph-599ee-6fb48cd747-x5wbq\" (UID: \"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8\") " pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:34:26.990605 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:34:26.990446 2577 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-599ee-serving-cert: secret "sequence-graph-599ee-serving-cert" not found Jul 06 21:34:26.990605 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:34:26.990554 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-proxy-tls podName:dbddb8a4-9f75-41f5-a1bd-db5e53852ca8 nodeName:}" failed. No retries permitted until 2026-07-06 21:34:27.490533821 +0000 UTC m=+1291.615515226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-proxy-tls") pod "sequence-graph-599ee-6fb48cd747-x5wbq" (UID: "dbddb8a4-9f75-41f5-a1bd-db5e53852ca8") : secret "sequence-graph-599ee-serving-cert" not found Jul 06 21:34:26.990960 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:26.990942 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-openshift-service-ca-bundle\") pod \"sequence-graph-599ee-6fb48cd747-x5wbq\" (UID: \"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8\") " pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:34:27.495066 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:27.495011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-proxy-tls\") pod \"sequence-graph-599ee-6fb48cd747-x5wbq\" (UID: \"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8\") " pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:34:27.495335 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:34:27.495168 2577 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-599ee-serving-cert: secret "sequence-graph-599ee-serving-cert" not found Jul 06 21:34:27.495335 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:34:27.495240 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-proxy-tls podName:dbddb8a4-9f75-41f5-a1bd-db5e53852ca8 nodeName:}" failed. No retries permitted until 2026-07-06 21:34:28.495226734 +0000 UTC m=+1292.620208135 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-proxy-tls") pod "sequence-graph-599ee-6fb48cd747-x5wbq" (UID: "dbddb8a4-9f75-41f5-a1bd-db5e53852ca8") : secret "sequence-graph-599ee-serving-cert" not found Jul 06 21:34:28.504401 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:28.504354 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-proxy-tls\") pod \"sequence-graph-599ee-6fb48cd747-x5wbq\" (UID: \"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8\") " pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:34:28.506869 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:28.506843 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-proxy-tls\") pod \"sequence-graph-599ee-6fb48cd747-x5wbq\" (UID: \"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8\") " pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:34:28.615516 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:28.615480 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:34:28.745298 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:28.745196 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq"] Jul 06 21:34:28.747346 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:34:28.747324 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbddb8a4_9f75_41f5_a1bd_db5e53852ca8.slice/crio-9d8ff17b4c5ba2b35591384da8ae6534527859784dc34ad8ce456c71791037eb WatchSource:0}: Error finding container 9d8ff17b4c5ba2b35591384da8ae6534527859784dc34ad8ce456c71791037eb: Status 404 returned error can't find the container with id 9d8ff17b4c5ba2b35591384da8ae6534527859784dc34ad8ce456c71791037eb Jul 06 21:34:28.814511 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:28.814481 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:28.992983 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:28.992935 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" event={"ID":"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8","Type":"ContainerStarted","Data":"0043d3b6078b7cb59295ce4d25847a8527819b5e0ee73575be08206c872b04fa"} Jul 06 21:34:28.993204 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:28.993045 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:34:28.993204 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:28.993068 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" event={"ID":"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8","Type":"ContainerStarted","Data":"9d8ff17b4c5ba2b35591384da8ae6534527859784dc34ad8ce456c71791037eb"} Jul 06 21:34:29.026851 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:29.026736 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" podStartSLOduration=3.026719283 podStartE2EDuration="3.026719283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:34:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:34:29.026137531 +0000 UTC m=+1293.151118953" watchObservedRunningTime="2026-07-06 21:34:29.026719283 +0000 UTC m=+1293.151700705" Jul 06 21:34:35.002815 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:35.002787 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:34:36.880890 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:36.880854 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq"] Jul 06 21:34:36.881269 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:36.881046 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" podUID="dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" containerName="sequence-graph-599ee" containerID="cri-o://0043d3b6078b7cb59295ce4d25847a8527819b5e0ee73575be08206c872b04fa" gracePeriod=30 Jul 06 21:34:36.910517 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:36.910488 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:37.022034 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.022005 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:37.122934 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.122905 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:37.200798 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.200719 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:37.205626 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.205593 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9"] Jul 06 21:34:37.209109 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.209092 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" Jul 06 21:34:37.213193 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.213170 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-9ded0-serving-cert\"" Jul 06 21:34:37.213329 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.213171 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-9ded0-kube-rbac-proxy-sar-config\"" Jul 06 21:34:37.221241 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.221217 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9"] Jul 06 21:34:37.270518 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.270483 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2684154-c96a-4dbf-8c5c-8b62a03c1789-openshift-service-ca-bundle\") pod \"ensemble-graph-9ded0-5b84bdf89c-htgg9\" (UID: \"e2684154-c96a-4dbf-8c5c-8b62a03c1789\") " pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" Jul 06 21:34:37.270680 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.270561 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2684154-c96a-4dbf-8c5c-8b62a03c1789-proxy-tls\") pod \"ensemble-graph-9ded0-5b84bdf89c-htgg9\" (UID: \"e2684154-c96a-4dbf-8c5c-8b62a03c1789\") " pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" Jul 06 21:34:37.298675 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.298644 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:37.371627 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.371599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2684154-c96a-4dbf-8c5c-8b62a03c1789-proxy-tls\") pod \"ensemble-graph-9ded0-5b84bdf89c-htgg9\" (UID: \"e2684154-c96a-4dbf-8c5c-8b62a03c1789\") " pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" Jul 06 21:34:37.371813 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.371652 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2684154-c96a-4dbf-8c5c-8b62a03c1789-openshift-service-ca-bundle\") pod \"ensemble-graph-9ded0-5b84bdf89c-htgg9\" (UID: \"e2684154-c96a-4dbf-8c5c-8b62a03c1789\") " pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" Jul 06 21:34:37.372313 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.372293 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2684154-c96a-4dbf-8c5c-8b62a03c1789-openshift-service-ca-bundle\") pod \"ensemble-graph-9ded0-5b84bdf89c-htgg9\" (UID: \"e2684154-c96a-4dbf-8c5c-8b62a03c1789\") " pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" Jul 06 21:34:37.373956 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.373930 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2684154-c96a-4dbf-8c5c-8b62a03c1789-proxy-tls\") pod \"ensemble-graph-9ded0-5b84bdf89c-htgg9\" (UID: \"e2684154-c96a-4dbf-8c5c-8b62a03c1789\") " pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" Jul 06 21:34:37.386756 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.386736 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:37.519537 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.519441 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" Jul 06 21:34:37.651631 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:37.651601 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9"] Jul 06 21:34:37.654791 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:34:37.654758 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2684154_c96a_4dbf_8c5c_8b62a03c1789.slice/crio-c9c48db4c97573725d641dfe1e543e4e09ded771c0103f4491cc02412a0e7123 WatchSource:0}: Error finding container c9c48db4c97573725d641dfe1e543e4e09ded771c0103f4491cc02412a0e7123: Status 404 returned error can't find the container with id c9c48db4c97573725d641dfe1e543e4e09ded771c0103f4491cc02412a0e7123 Jul 06 21:34:38.029934 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:38.029897 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" event={"ID":"e2684154-c96a-4dbf-8c5c-8b62a03c1789","Type":"ContainerStarted","Data":"414e9b23509b56d23ec0e8f15e2dbebd5598433d7aa2bf8cd85128082ce6627e"} Jul 06 21:34:38.030285 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:38.029952 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" event={"ID":"e2684154-c96a-4dbf-8c5c-8b62a03c1789","Type":"ContainerStarted","Data":"c9c48db4c97573725d641dfe1e543e4e09ded771c0103f4491cc02412a0e7123"} Jul 06 21:34:38.030285 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:38.030034 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" Jul 06 21:34:38.049378 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:38.049330 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" podStartSLOduration=1.049315074 podStartE2EDuration="1.049315074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:34:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:34:38.048202575 +0000 UTC m=+1302.173183990" watchObservedRunningTime="2026-07-06 21:34:38.049315074 +0000 UTC m=+1302.174296496" Jul 06 21:34:40.001576 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:40.001535 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" podUID="dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" containerName="sequence-graph-599ee" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:34:44.041337 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:44.041304 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" Jul 06 21:34:45.000932 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:45.000892 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" podUID="dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" containerName="sequence-graph-599ee" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:34:50.001372 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:50.001331 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" podUID="dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" containerName="sequence-graph-599ee" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:34:50.001769 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:50.001434 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:34:55.001351 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:34:55.001314 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" podUID="dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" containerName="sequence-graph-599ee" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:35:00.001448 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:00.001406 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" podUID="dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" containerName="sequence-graph-599ee" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:35:05.001594 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:05.001554 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" podUID="dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" containerName="sequence-graph-599ee" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:35:07.064593 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.064568 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:35:07.162738 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.162643 2577 generic.go:356] "Generic (PLEG): container finished" podID="dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" containerID="0043d3b6078b7cb59295ce4d25847a8527819b5e0ee73575be08206c872b04fa" exitCode=0 Jul 06 21:35:07.162738 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.162708 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" Jul 06 21:35:07.162913 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.162731 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" event={"ID":"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8","Type":"ContainerDied","Data":"0043d3b6078b7cb59295ce4d25847a8527819b5e0ee73575be08206c872b04fa"} Jul 06 21:35:07.162913 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.162810 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq" event={"ID":"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8","Type":"ContainerDied","Data":"9d8ff17b4c5ba2b35591384da8ae6534527859784dc34ad8ce456c71791037eb"} Jul 06 21:35:07.162913 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.162836 2577 scope.go:117] "RemoveContainer" containerID="0043d3b6078b7cb59295ce4d25847a8527819b5e0ee73575be08206c872b04fa" Jul 06 21:35:07.170827 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.170811 2577 scope.go:117] "RemoveContainer" containerID="0043d3b6078b7cb59295ce4d25847a8527819b5e0ee73575be08206c872b04fa" Jul 06 21:35:07.171091 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:35:07.171074 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0043d3b6078b7cb59295ce4d25847a8527819b5e0ee73575be08206c872b04fa\": container with ID starting with 0043d3b6078b7cb59295ce4d25847a8527819b5e0ee73575be08206c872b04fa not found: ID does not exist" containerID="0043d3b6078b7cb59295ce4d25847a8527819b5e0ee73575be08206c872b04fa" Jul 06 21:35:07.171133 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.171099 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0043d3b6078b7cb59295ce4d25847a8527819b5e0ee73575be08206c872b04fa"} err="failed to get container status \"0043d3b6078b7cb59295ce4d25847a8527819b5e0ee73575be08206c872b04fa\": rpc error: code = NotFound desc = could not find container \"0043d3b6078b7cb59295ce4d25847a8527819b5e0ee73575be08206c872b04fa\": container with ID starting with 0043d3b6078b7cb59295ce4d25847a8527819b5e0ee73575be08206c872b04fa not found: ID does not exist" Jul 06 21:35:07.194355 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.194329 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-proxy-tls\") pod \"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8\" (UID: \"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8\") " Jul 06 21:35:07.194449 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.194396 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-openshift-service-ca-bundle\") pod \"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8\" (UID: \"dbddb8a4-9f75-41f5-a1bd-db5e53852ca8\") " Jul 06 21:35:07.194754 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.194735 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" (UID: "dbddb8a4-9f75-41f5-a1bd-db5e53852ca8"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:35:07.196320 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.196297 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" (UID: "dbddb8a4-9f75-41f5-a1bd-db5e53852ca8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:35:07.295212 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.295173 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-openshift-service-ca-bundle\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:35:07.295212 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.295206 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8-proxy-tls\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:35:07.483435 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.483405 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq"] Jul 06 21:35:07.489863 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:07.489839 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-599ee-6fb48cd747-x5wbq"] Jul 06 21:35:08.489026 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:08.488991 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" path="/var/lib/kubelet/pods/dbddb8a4-9f75-41f5-a1bd-db5e53852ca8/volumes" Jul 06 21:35:37.105067 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.105027 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw"] Jul 06 21:35:37.105672 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.105543 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" containerName="sequence-graph-599ee" Jul 06 21:35:37.105672 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.105563 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" containerName="sequence-graph-599ee" Jul 06 21:35:37.105672 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.105668 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbddb8a4-9f75-41f5-a1bd-db5e53852ca8" containerName="sequence-graph-599ee" Jul 06 21:35:37.108555 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.108534 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" Jul 06 21:35:37.111389 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.111364 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-a54dc-kube-rbac-proxy-sar-config\"" Jul 06 21:35:37.111537 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.111426 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-a54dc-serving-cert\"" Jul 06 21:35:37.119969 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.119942 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw"] Jul 06 21:35:37.120092 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.119941 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:35:37.216919 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.216877 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:35:37.232746 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.232701 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86b14b6b-2f46-4b3c-87a1-4634f94b2160-openshift-service-ca-bundle\") pod \"sequence-graph-a54dc-84c7c69c68-5ndvw\" (UID: \"86b14b6b-2f46-4b3c-87a1-4634f94b2160\") " pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" Jul 06 21:35:37.232923 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.232821 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86b14b6b-2f46-4b3c-87a1-4634f94b2160-proxy-tls\") pod \"sequence-graph-a54dc-84c7c69c68-5ndvw\" (UID: \"86b14b6b-2f46-4b3c-87a1-4634f94b2160\") " pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" Jul 06 21:35:37.333777 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.333738 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86b14b6b-2f46-4b3c-87a1-4634f94b2160-openshift-service-ca-bundle\") pod \"sequence-graph-a54dc-84c7c69c68-5ndvw\" (UID: \"86b14b6b-2f46-4b3c-87a1-4634f94b2160\") " pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" Jul 06 21:35:37.333926 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.333835 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86b14b6b-2f46-4b3c-87a1-4634f94b2160-proxy-tls\") pod \"sequence-graph-a54dc-84c7c69c68-5ndvw\" (UID: \"86b14b6b-2f46-4b3c-87a1-4634f94b2160\") " pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" Jul 06 21:35:37.334357 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.334337 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86b14b6b-2f46-4b3c-87a1-4634f94b2160-openshift-service-ca-bundle\") pod \"sequence-graph-a54dc-84c7c69c68-5ndvw\" (UID: \"86b14b6b-2f46-4b3c-87a1-4634f94b2160\") " pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" Jul 06 21:35:37.336194 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.336168 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86b14b6b-2f46-4b3c-87a1-4634f94b2160-proxy-tls\") pod \"sequence-graph-a54dc-84c7c69c68-5ndvw\" (UID: \"86b14b6b-2f46-4b3c-87a1-4634f94b2160\") " pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" Jul 06 21:35:37.419695 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.419596 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" Jul 06 21:35:37.549514 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:37.549446 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw"] Jul 06 21:35:37.551920 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:35:37.551891 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86b14b6b_2f46_4b3c_87a1_4634f94b2160.slice/crio-86ce84052e9510b70ac4353d36809d72a937313ffd3a8a0f6f874ddf785b49d8 WatchSource:0}: Error finding container 86ce84052e9510b70ac4353d36809d72a937313ffd3a8a0f6f874ddf785b49d8: Status 404 returned error can't find the container with id 86ce84052e9510b70ac4353d36809d72a937313ffd3a8a0f6f874ddf785b49d8 Jul 06 21:35:38.296487 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:38.296425 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" event={"ID":"86b14b6b-2f46-4b3c-87a1-4634f94b2160","Type":"ContainerStarted","Data":"58589488771cb3002a7989ff2903ccbb446ef96951a57be026e65606f0c1f300"} Jul 06 21:35:38.296883 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:38.296513 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" Jul 06 21:35:38.296883 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:38.296538 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" event={"ID":"86b14b6b-2f46-4b3c-87a1-4634f94b2160","Type":"ContainerStarted","Data":"86ce84052e9510b70ac4353d36809d72a937313ffd3a8a0f6f874ddf785b49d8"} Jul 06 21:35:38.328418 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:38.328365 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" podStartSLOduration=1.32834911 podStartE2EDuration="1.32834911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:35:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:35:38.326193482 +0000 UTC m=+1362.451174908" watchObservedRunningTime="2026-07-06 21:35:38.32834911 +0000 UTC m=+1362.453330532" Jul 06 21:35:44.305901 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:35:44.305871 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" Jul 06 21:37:56.394290 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:37:56.394263 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:37:56.397136 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:37:56.397110 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:37:56.397817 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:37:56.397799 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:37:56.400372 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:37:56.400353 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:38:03.830594 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:38:03.830558 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:38:27.679716 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:38:27.679630 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:40:01.087545 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:40:01.087509 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:42:51.855266 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:51.855180 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9"] Jul 06 21:42:51.855836 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:51.855527 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" podUID="e2684154-c96a-4dbf-8c5c-8b62a03c1789" containerName="ensemble-graph-9ded0" containerID="cri-o://414e9b23509b56d23ec0e8f15e2dbebd5598433d7aa2bf8cd85128082ce6627e" gracePeriod=30 Jul 06 21:42:51.912342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:51.912311 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:42:52.009776 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:52.009743 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:42:52.108195 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:52.108117 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:42:52.181733 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:52.181697 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:42:52.280856 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:52.280818 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:42:52.382372 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:52.382290 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:42:54.038824 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:54.038789 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" podUID="e2684154-c96a-4dbf-8c5c-8b62a03c1789" containerName="ensemble-graph-9ded0" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:42:56.420712 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:56.420680 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:42:56.422486 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:56.422442 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:42:56.424177 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:56.424157 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:42:56.426023 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:56.425998 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:42:59.038519 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:42:59.038455 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" podUID="e2684154-c96a-4dbf-8c5c-8b62a03c1789" containerName="ensemble-graph-9ded0" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:43:04.038771 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:04.038722 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" podUID="e2684154-c96a-4dbf-8c5c-8b62a03c1789" containerName="ensemble-graph-9ded0" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:43:04.039229 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:04.038904 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" Jul 06 21:43:09.038420 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:09.038372 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" podUID="e2684154-c96a-4dbf-8c5c-8b62a03c1789" containerName="ensemble-graph-9ded0" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:43:14.040500 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:14.040430 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" podUID="e2684154-c96a-4dbf-8c5c-8b62a03c1789" containerName="ensemble-graph-9ded0" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:43:19.038940 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:19.038903 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" podUID="e2684154-c96a-4dbf-8c5c-8b62a03c1789" containerName="ensemble-graph-9ded0" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:43:22.031708 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.031680 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" Jul 06 21:43:22.107850 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.107805 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2684154-c96a-4dbf-8c5c-8b62a03c1789-openshift-service-ca-bundle\") pod \"e2684154-c96a-4dbf-8c5c-8b62a03c1789\" (UID: \"e2684154-c96a-4dbf-8c5c-8b62a03c1789\") " Jul 06 21:43:22.108039 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.107875 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2684154-c96a-4dbf-8c5c-8b62a03c1789-proxy-tls\") pod \"e2684154-c96a-4dbf-8c5c-8b62a03c1789\" (UID: \"e2684154-c96a-4dbf-8c5c-8b62a03c1789\") " Jul 06 21:43:22.108155 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.108132 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2684154-c96a-4dbf-8c5c-8b62a03c1789-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "e2684154-c96a-4dbf-8c5c-8b62a03c1789" (UID: "e2684154-c96a-4dbf-8c5c-8b62a03c1789"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:43:22.109929 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.109878 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2684154-c96a-4dbf-8c5c-8b62a03c1789-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e2684154-c96a-4dbf-8c5c-8b62a03c1789" (UID: "e2684154-c96a-4dbf-8c5c-8b62a03c1789"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:43:22.208415 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.208373 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2684154-c96a-4dbf-8c5c-8b62a03c1789-openshift-service-ca-bundle\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:43:22.208415 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.208409 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2684154-c96a-4dbf-8c5c-8b62a03c1789-proxy-tls\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:43:22.264408 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.264377 2577 generic.go:356] "Generic (PLEG): container finished" podID="e2684154-c96a-4dbf-8c5c-8b62a03c1789" containerID="414e9b23509b56d23ec0e8f15e2dbebd5598433d7aa2bf8cd85128082ce6627e" exitCode=0 Jul 06 21:43:22.264562 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.264424 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" Jul 06 21:43:22.264562 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.264418 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" event={"ID":"e2684154-c96a-4dbf-8c5c-8b62a03c1789","Type":"ContainerDied","Data":"414e9b23509b56d23ec0e8f15e2dbebd5598433d7aa2bf8cd85128082ce6627e"} Jul 06 21:43:22.264638 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.264565 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9" event={"ID":"e2684154-c96a-4dbf-8c5c-8b62a03c1789","Type":"ContainerDied","Data":"c9c48db4c97573725d641dfe1e543e4e09ded771c0103f4491cc02412a0e7123"} Jul 06 21:43:22.264638 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.264594 2577 scope.go:117] "RemoveContainer" containerID="414e9b23509b56d23ec0e8f15e2dbebd5598433d7aa2bf8cd85128082ce6627e" Jul 06 21:43:22.273077 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.273055 2577 scope.go:117] "RemoveContainer" containerID="414e9b23509b56d23ec0e8f15e2dbebd5598433d7aa2bf8cd85128082ce6627e" Jul 06 21:43:22.273320 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:43:22.273300 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"414e9b23509b56d23ec0e8f15e2dbebd5598433d7aa2bf8cd85128082ce6627e\": container with ID starting with 414e9b23509b56d23ec0e8f15e2dbebd5598433d7aa2bf8cd85128082ce6627e not found: ID does not exist" containerID="414e9b23509b56d23ec0e8f15e2dbebd5598433d7aa2bf8cd85128082ce6627e" Jul 06 21:43:22.273372 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.273329 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"414e9b23509b56d23ec0e8f15e2dbebd5598433d7aa2bf8cd85128082ce6627e"} err="failed to get container status \"414e9b23509b56d23ec0e8f15e2dbebd5598433d7aa2bf8cd85128082ce6627e\": rpc error: code = NotFound desc = could not find container \"414e9b23509b56d23ec0e8f15e2dbebd5598433d7aa2bf8cd85128082ce6627e\": container with ID starting with 414e9b23509b56d23ec0e8f15e2dbebd5598433d7aa2bf8cd85128082ce6627e not found: ID does not exist" Jul 06 21:43:22.284784 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.284758 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9"] Jul 06 21:43:22.286884 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.286861 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-9ded0-5b84bdf89c-htgg9"] Jul 06 21:43:22.489474 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:22.489384 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2684154-c96a-4dbf-8c5c-8b62a03c1789" path="/var/lib/kubelet/pods/e2684154-c96a-4dbf-8c5c-8b62a03c1789/volumes" Jul 06 21:43:51.795936 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:51.795900 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw"] Jul 06 21:43:51.796629 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:51.796115 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" podUID="86b14b6b-2f46-4b3c-87a1-4634f94b2160" containerName="sequence-graph-a54dc" containerID="cri-o://58589488771cb3002a7989ff2903ccbb446ef96951a57be026e65606f0c1f300" gracePeriod=30 Jul 06 21:43:51.826122 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:51.826090 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:43:51.915707 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:51.915666 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:43:52.002709 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.002678 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:43:52.069478 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.069396 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:43:52.074048 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.074022 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr"] Jul 06 21:43:52.074372 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.074360 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2684154-c96a-4dbf-8c5c-8b62a03c1789" containerName="ensemble-graph-9ded0" Jul 06 21:43:52.074413 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.074374 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2684154-c96a-4dbf-8c5c-8b62a03c1789" containerName="ensemble-graph-9ded0" Jul 06 21:43:52.074446 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.074424 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2684154-c96a-4dbf-8c5c-8b62a03c1789" containerName="ensemble-graph-9ded0" Jul 06 21:43:52.077233 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.077216 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" Jul 06 21:43:52.079689 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.079663 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-974ed-kube-rbac-proxy-sar-config\"" Jul 06 21:43:52.079776 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.079721 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-974ed-serving-cert\"" Jul 06 21:43:52.088260 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.088236 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr"] Jul 06 21:43:52.170179 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.170146 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:43:52.255546 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.255502 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b279ff55-03bc-4eab-864c-ca994cba9912-openshift-service-ca-bundle\") pod \"splitter-graph-974ed-d89cd458-jn6tr\" (UID: \"b279ff55-03bc-4eab-864c-ca994cba9912\") " pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" Jul 06 21:43:52.255723 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.255591 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b279ff55-03bc-4eab-864c-ca994cba9912-proxy-tls\") pod \"splitter-graph-974ed-d89cd458-jn6tr\" (UID: \"b279ff55-03bc-4eab-864c-ca994cba9912\") " pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" Jul 06 21:43:52.270952 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.270921 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:43:52.329322 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.329297 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:43:52.356426 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.356406 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b279ff55-03bc-4eab-864c-ca994cba9912-openshift-service-ca-bundle\") pod \"splitter-graph-974ed-d89cd458-jn6tr\" (UID: \"b279ff55-03bc-4eab-864c-ca994cba9912\") " pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" Jul 06 21:43:52.356542 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.356457 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b279ff55-03bc-4eab-864c-ca994cba9912-proxy-tls\") pod \"splitter-graph-974ed-d89cd458-jn6tr\" (UID: \"b279ff55-03bc-4eab-864c-ca994cba9912\") " pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" Jul 06 21:43:52.357025 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.357008 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b279ff55-03bc-4eab-864c-ca994cba9912-openshift-service-ca-bundle\") pod \"splitter-graph-974ed-d89cd458-jn6tr\" (UID: \"b279ff55-03bc-4eab-864c-ca994cba9912\") " pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" Jul 06 21:43:52.358941 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.358925 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b279ff55-03bc-4eab-864c-ca994cba9912-proxy-tls\") pod \"splitter-graph-974ed-d89cd458-jn6tr\" (UID: \"b279ff55-03bc-4eab-864c-ca994cba9912\") " pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" Jul 06 21:43:52.388046 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.387999 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" Jul 06 21:43:52.519981 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:52.519924 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr"] Jul 06 21:43:52.522279 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:43:52.522249 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb279ff55_03bc_4eab_864c_ca994cba9912.slice/crio-34536454b48aee7c7db7006498e477550d30eb96f89f2b05aaf6ca42719c0216 WatchSource:0}: Error finding container 34536454b48aee7c7db7006498e477550d30eb96f89f2b05aaf6ca42719c0216: Status 404 returned error can't find the container with id 34536454b48aee7c7db7006498e477550d30eb96f89f2b05aaf6ca42719c0216 Jul 06 21:43:53.403307 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:53.403272 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" event={"ID":"b279ff55-03bc-4eab-864c-ca994cba9912","Type":"ContainerStarted","Data":"ca1a2c5c4cd59edec9d031d7cb03e812a418ca756859c022945a4081306cc309"} Jul 06 21:43:53.403718 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:53.403324 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" event={"ID":"b279ff55-03bc-4eab-864c-ca994cba9912","Type":"ContainerStarted","Data":"34536454b48aee7c7db7006498e477550d30eb96f89f2b05aaf6ca42719c0216"} Jul 06 21:43:53.403718 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:53.403398 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" Jul 06 21:43:53.419558 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:53.419502 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" podStartSLOduration=1.419488515 podStartE2EDuration="1.419488515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:43:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:43:53.417857524 +0000 UTC m=+1857.542838946" watchObservedRunningTime="2026-07-06 21:43:53.419488515 +0000 UTC m=+1857.544469934" Jul 06 21:43:54.304924 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:54.304877 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" podUID="86b14b6b-2f46-4b3c-87a1-4634f94b2160" containerName="sequence-graph-a54dc" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:43:59.304613 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:59.304570 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" podUID="86b14b6b-2f46-4b3c-87a1-4634f94b2160" containerName="sequence-graph-a54dc" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:43:59.412988 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:43:59.412960 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" Jul 06 21:44:02.184955 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:02.184918 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr"] Jul 06 21:44:02.185425 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:02.185139 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" podUID="b279ff55-03bc-4eab-864c-ca994cba9912" containerName="splitter-graph-974ed" containerID="cri-o://ca1a2c5c4cd59edec9d031d7cb03e812a418ca756859c022945a4081306cc309" gracePeriod=30 Jul 06 21:44:02.213771 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:02.213732 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:44:02.350050 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:02.350014 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:44:02.449691 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:02.449615 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:44:02.545612 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:02.545576 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:44:02.791980 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:02.791906 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:44:03.059198 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:03.059125 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:44:04.304458 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:04.304421 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" podUID="86b14b6b-2f46-4b3c-87a1-4634f94b2160" containerName="sequence-graph-a54dc" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:44:04.304847 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:04.304562 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" Jul 06 21:44:04.411408 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:04.411366 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" podUID="b279ff55-03bc-4eab-864c-ca994cba9912" containerName="splitter-graph-974ed" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:44:09.305145 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:09.305105 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" podUID="86b14b6b-2f46-4b3c-87a1-4634f94b2160" containerName="sequence-graph-a54dc" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:44:09.412063 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:09.412016 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" podUID="b279ff55-03bc-4eab-864c-ca994cba9912" containerName="splitter-graph-974ed" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:44:14.304826 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:14.304782 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" podUID="86b14b6b-2f46-4b3c-87a1-4634f94b2160" containerName="sequence-graph-a54dc" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:44:14.411992 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:14.411950 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" podUID="b279ff55-03bc-4eab-864c-ca994cba9912" containerName="splitter-graph-974ed" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:44:14.412178 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:14.412080 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" Jul 06 21:44:19.305619 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:19.305523 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" podUID="86b14b6b-2f46-4b3c-87a1-4634f94b2160" containerName="sequence-graph-a54dc" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:44:19.411364 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:19.411325 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" podUID="b279ff55-03bc-4eab-864c-ca994cba9912" containerName="splitter-graph-974ed" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:44:22.445305 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.445273 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" Jul 06 21:44:22.477699 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.477669 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86b14b6b-2f46-4b3c-87a1-4634f94b2160-openshift-service-ca-bundle\") pod \"86b14b6b-2f46-4b3c-87a1-4634f94b2160\" (UID: \"86b14b6b-2f46-4b3c-87a1-4634f94b2160\") " Jul 06 21:44:22.477847 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.477729 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86b14b6b-2f46-4b3c-87a1-4634f94b2160-proxy-tls\") pod \"86b14b6b-2f46-4b3c-87a1-4634f94b2160\" (UID: \"86b14b6b-2f46-4b3c-87a1-4634f94b2160\") " Jul 06 21:44:22.478046 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.478022 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86b14b6b-2f46-4b3c-87a1-4634f94b2160-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "86b14b6b-2f46-4b3c-87a1-4634f94b2160" (UID: "86b14b6b-2f46-4b3c-87a1-4634f94b2160"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:44:22.479814 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.479789 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86b14b6b-2f46-4b3c-87a1-4634f94b2160-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "86b14b6b-2f46-4b3c-87a1-4634f94b2160" (UID: "86b14b6b-2f46-4b3c-87a1-4634f94b2160"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:44:22.522701 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.522667 2577 generic.go:356] "Generic (PLEG): container finished" podID="86b14b6b-2f46-4b3c-87a1-4634f94b2160" containerID="58589488771cb3002a7989ff2903ccbb446ef96951a57be026e65606f0c1f300" exitCode=0 Jul 06 21:44:22.522858 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.522710 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" Jul 06 21:44:22.522858 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.522740 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" event={"ID":"86b14b6b-2f46-4b3c-87a1-4634f94b2160","Type":"ContainerDied","Data":"58589488771cb3002a7989ff2903ccbb446ef96951a57be026e65606f0c1f300"} Jul 06 21:44:22.522858 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.522793 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw" event={"ID":"86b14b6b-2f46-4b3c-87a1-4634f94b2160","Type":"ContainerDied","Data":"86ce84052e9510b70ac4353d36809d72a937313ffd3a8a0f6f874ddf785b49d8"} Jul 06 21:44:22.523418 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.522815 2577 scope.go:117] "RemoveContainer" containerID="58589488771cb3002a7989ff2903ccbb446ef96951a57be026e65606f0c1f300" Jul 06 21:44:22.531724 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.531707 2577 scope.go:117] "RemoveContainer" containerID="58589488771cb3002a7989ff2903ccbb446ef96951a57be026e65606f0c1f300" Jul 06 21:44:22.531976 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:44:22.531956 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58589488771cb3002a7989ff2903ccbb446ef96951a57be026e65606f0c1f300\": container with ID starting with 58589488771cb3002a7989ff2903ccbb446ef96951a57be026e65606f0c1f300 not found: ID does not exist" containerID="58589488771cb3002a7989ff2903ccbb446ef96951a57be026e65606f0c1f300" Jul 06 21:44:22.532020 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.531985 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58589488771cb3002a7989ff2903ccbb446ef96951a57be026e65606f0c1f300"} err="failed to get container status \"58589488771cb3002a7989ff2903ccbb446ef96951a57be026e65606f0c1f300\": rpc error: code = NotFound desc = could not find container \"58589488771cb3002a7989ff2903ccbb446ef96951a57be026e65606f0c1f300\": container with ID starting with 58589488771cb3002a7989ff2903ccbb446ef96951a57be026e65606f0c1f300 not found: ID does not exist" Jul 06 21:44:22.538126 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.538103 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw"] Jul 06 21:44:22.542439 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.542419 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-a54dc-84c7c69c68-5ndvw"] Jul 06 21:44:22.578364 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.578323 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86b14b6b-2f46-4b3c-87a1-4634f94b2160-openshift-service-ca-bundle\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:44:22.578364 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:22.578353 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86b14b6b-2f46-4b3c-87a1-4634f94b2160-proxy-tls\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:44:24.411378 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:24.411334 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" podUID="b279ff55-03bc-4eab-864c-ca994cba9912" containerName="splitter-graph-974ed" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:44:24.489230 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:24.489201 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86b14b6b-2f46-4b3c-87a1-4634f94b2160" path="/var/lib/kubelet/pods/86b14b6b-2f46-4b3c-87a1-4634f94b2160/volumes" Jul 06 21:44:29.412048 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:29.412004 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" podUID="b279ff55-03bc-4eab-864c-ca994cba9912" containerName="splitter-graph-974ed" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:44:32.568175 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:32.568084 2577 generic.go:356] "Generic (PLEG): container finished" podID="b279ff55-03bc-4eab-864c-ca994cba9912" containerID="ca1a2c5c4cd59edec9d031d7cb03e812a418ca756859c022945a4081306cc309" exitCode=0 Jul 06 21:44:32.568581 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:32.568161 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" event={"ID":"b279ff55-03bc-4eab-864c-ca994cba9912","Type":"ContainerDied","Data":"ca1a2c5c4cd59edec9d031d7cb03e812a418ca756859c022945a4081306cc309"} Jul 06 21:44:32.843342 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:32.843317 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" Jul 06 21:44:32.961755 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:32.961724 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b279ff55-03bc-4eab-864c-ca994cba9912-proxy-tls\") pod \"b279ff55-03bc-4eab-864c-ca994cba9912\" (UID: \"b279ff55-03bc-4eab-864c-ca994cba9912\") " Jul 06 21:44:32.961937 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:32.961761 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b279ff55-03bc-4eab-864c-ca994cba9912-openshift-service-ca-bundle\") pod \"b279ff55-03bc-4eab-864c-ca994cba9912\" (UID: \"b279ff55-03bc-4eab-864c-ca994cba9912\") " Jul 06 21:44:32.962135 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:32.962103 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b279ff55-03bc-4eab-864c-ca994cba9912-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b279ff55-03bc-4eab-864c-ca994cba9912" (UID: "b279ff55-03bc-4eab-864c-ca994cba9912"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:44:32.963825 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:32.963803 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b279ff55-03bc-4eab-864c-ca994cba9912-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b279ff55-03bc-4eab-864c-ca994cba9912" (UID: "b279ff55-03bc-4eab-864c-ca994cba9912"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:44:33.062331 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:33.062295 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b279ff55-03bc-4eab-864c-ca994cba9912-proxy-tls\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:44:33.062331 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:33.062325 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b279ff55-03bc-4eab-864c-ca994cba9912-openshift-service-ca-bundle\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:44:33.574370 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:33.574338 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" Jul 06 21:44:33.574898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:33.574336 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr" event={"ID":"b279ff55-03bc-4eab-864c-ca994cba9912","Type":"ContainerDied","Data":"34536454b48aee7c7db7006498e477550d30eb96f89f2b05aaf6ca42719c0216"} Jul 06 21:44:33.574898 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:33.574511 2577 scope.go:117] "RemoveContainer" containerID="ca1a2c5c4cd59edec9d031d7cb03e812a418ca756859c022945a4081306cc309" Jul 06 21:44:33.594977 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:33.594949 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr"] Jul 06 21:44:33.600814 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:33.600795 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-974ed-d89cd458-jn6tr"] Jul 06 21:44:34.489270 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:44:34.489229 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b279ff55-03bc-4eab-864c-ca994cba9912" path="/var/lib/kubelet/pods/b279ff55-03bc-4eab-864c-ca994cba9912/volumes" Jul 06 21:45:01.997479 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:01.997438 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj"] Jul 06 21:45:01.999714 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:01.997776 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b279ff55-03bc-4eab-864c-ca994cba9912" containerName="splitter-graph-974ed" Jul 06 21:45:01.999714 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:01.997788 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b279ff55-03bc-4eab-864c-ca994cba9912" containerName="splitter-graph-974ed" Jul 06 21:45:01.999714 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:01.997809 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86b14b6b-2f46-4b3c-87a1-4634f94b2160" containerName="sequence-graph-a54dc" Jul 06 21:45:01.999714 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:01.997815 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="86b14b6b-2f46-4b3c-87a1-4634f94b2160" containerName="sequence-graph-a54dc" Jul 06 21:45:01.999714 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:01.997863 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="86b14b6b-2f46-4b3c-87a1-4634f94b2160" containerName="sequence-graph-a54dc" Jul 06 21:45:01.999714 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:01.997873 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b279ff55-03bc-4eab-864c-ca994cba9912" containerName="splitter-graph-974ed" Jul 06 21:45:02.000667 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.000646 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 21:45:02.003126 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.003098 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-da1f6-serving-cert\"" Jul 06 21:45:02.003228 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.003124 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2qpmz\"" Jul 06 21:45:02.003228 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.003098 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-da1f6-kube-rbac-proxy-sar-config\"" Jul 06 21:45:02.004065 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.004044 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 06 21:45:02.009575 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.009550 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj"] Jul 06 21:45:02.028481 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.028437 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:45:02.098986 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.098946 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-proxy-tls\") pod \"switch-graph-da1f6-ccc959c7-b2xwj\" (UID: \"8fb96fa6-d5fa-40b2-b61e-c079cf778c50\") " pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 21:45:02.099158 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.098999 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-openshift-service-ca-bundle\") pod \"switch-graph-da1f6-ccc959c7-b2xwj\" (UID: \"8fb96fa6-d5fa-40b2-b61e-c079cf778c50\") " pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 21:45:02.200054 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.200015 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-proxy-tls\") pod \"switch-graph-da1f6-ccc959c7-b2xwj\" (UID: \"8fb96fa6-d5fa-40b2-b61e-c079cf778c50\") " pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 21:45:02.200232 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.200064 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-openshift-service-ca-bundle\") pod \"switch-graph-da1f6-ccc959c7-b2xwj\" (UID: \"8fb96fa6-d5fa-40b2-b61e-c079cf778c50\") " pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 21:45:02.200232 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:45:02.200184 2577 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-da1f6-serving-cert: secret "switch-graph-da1f6-serving-cert" not found Jul 06 21:45:02.200311 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:45:02.200247 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-proxy-tls podName:8fb96fa6-d5fa-40b2-b61e-c079cf778c50 nodeName:}" failed. No retries permitted until 2026-07-06 21:45:02.70023093 +0000 UTC m=+1926.825212336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-proxy-tls") pod "switch-graph-da1f6-ccc959c7-b2xwj" (UID: "8fb96fa6-d5fa-40b2-b61e-c079cf778c50") : secret "switch-graph-da1f6-serving-cert" not found Jul 06 21:45:02.200674 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.200657 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-openshift-service-ca-bundle\") pod \"switch-graph-da1f6-ccc959c7-b2xwj\" (UID: \"8fb96fa6-d5fa-40b2-b61e-c079cf778c50\") " pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 21:45:02.277256 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.277178 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:45:02.704288 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.704253 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-proxy-tls\") pod \"switch-graph-da1f6-ccc959c7-b2xwj\" (UID: \"8fb96fa6-d5fa-40b2-b61e-c079cf778c50\") " pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 21:45:02.706720 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.706689 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-proxy-tls\") pod \"switch-graph-da1f6-ccc959c7-b2xwj\" (UID: \"8fb96fa6-d5fa-40b2-b61e-c079cf778c50\") " pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 21:45:02.911774 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:02.911723 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 21:45:03.043593 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:03.043568 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj"] Jul 06 21:45:03.045571 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:45:03.045543 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fb96fa6_d5fa_40b2_b61e_c079cf778c50.slice/crio-6eacf5a27bb67b7badad16aa1adc783834e4ad99268f188b11a6f614ed11347a WatchSource:0}: Error finding container 6eacf5a27bb67b7badad16aa1adc783834e4ad99268f188b11a6f614ed11347a: Status 404 returned error can't find the container with id 6eacf5a27bb67b7badad16aa1adc783834e4ad99268f188b11a6f614ed11347a Jul 06 21:45:03.700425 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:03.700384 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" event={"ID":"8fb96fa6-d5fa-40b2-b61e-c079cf778c50","Type":"ContainerStarted","Data":"a3a36a493cf15d7ea50adc5821ee3af2ce9e545b2aa4ea7b0708e8be427ba333"} Jul 06 21:45:03.700616 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:03.700495 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" event={"ID":"8fb96fa6-d5fa-40b2-b61e-c079cf778c50","Type":"ContainerStarted","Data":"6eacf5a27bb67b7badad16aa1adc783834e4ad99268f188b11a6f614ed11347a"} Jul 06 21:45:03.700616 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:03.700526 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 21:45:03.717018 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:03.716961 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" podStartSLOduration=2.716944284 podStartE2EDuration="2.716944284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:45:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:45:03.715159665 +0000 UTC m=+1927.840141106" watchObservedRunningTime="2026-07-06 21:45:03.716944284 +0000 UTC m=+1927.841925717" Jul 06 21:45:09.710871 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:09.710838 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 21:45:12.427747 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.427714 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j"] Jul 06 21:45:12.431111 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.431085 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" Jul 06 21:45:12.433454 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.433429 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-563cb-kube-rbac-proxy-sar-config\"" Jul 06 21:45:12.433583 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.433438 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-563cb-serving-cert\"" Jul 06 21:45:12.439426 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.439402 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:45:12.440312 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.440290 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j"] Jul 06 21:45:12.585168 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.585130 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e391781d-e9ed-4f8a-8a36-72e3526f2914-proxy-tls\") pod \"splitter-graph-563cb-656565cb55-rbk2j\" (UID: \"e391781d-e9ed-4f8a-8a36-72e3526f2914\") " pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" Jul 06 21:45:12.585396 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.585176 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e391781d-e9ed-4f8a-8a36-72e3526f2914-openshift-service-ca-bundle\") pod \"splitter-graph-563cb-656565cb55-rbk2j\" (UID: \"e391781d-e9ed-4f8a-8a36-72e3526f2914\") " pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" Jul 06 21:45:12.686564 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.686458 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e391781d-e9ed-4f8a-8a36-72e3526f2914-openshift-service-ca-bundle\") pod \"splitter-graph-563cb-656565cb55-rbk2j\" (UID: \"e391781d-e9ed-4f8a-8a36-72e3526f2914\") " pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" Jul 06 21:45:12.686710 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.686573 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e391781d-e9ed-4f8a-8a36-72e3526f2914-proxy-tls\") pod \"splitter-graph-563cb-656565cb55-rbk2j\" (UID: \"e391781d-e9ed-4f8a-8a36-72e3526f2914\") " pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" Jul 06 21:45:12.687068 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.687041 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e391781d-e9ed-4f8a-8a36-72e3526f2914-openshift-service-ca-bundle\") pod \"splitter-graph-563cb-656565cb55-rbk2j\" (UID: \"e391781d-e9ed-4f8a-8a36-72e3526f2914\") " pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" Jul 06 21:45:12.688988 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.688964 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e391781d-e9ed-4f8a-8a36-72e3526f2914-proxy-tls\") pod \"splitter-graph-563cb-656565cb55-rbk2j\" (UID: \"e391781d-e9ed-4f8a-8a36-72e3526f2914\") " pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" Jul 06 21:45:12.742371 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.742339 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" Jul 06 21:45:12.870607 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:12.870581 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j"] Jul 06 21:45:12.872830 ip-10-0-136-12 kubenswrapper[2577]: W0706 21:45:12.872802 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode391781d_e9ed_4f8a_8a36_72e3526f2914.slice/crio-1e433bfa4a022e6f455754dae4fe1c4583a875bbe5b1414997b949c0caf5dd5e WatchSource:0}: Error finding container 1e433bfa4a022e6f455754dae4fe1c4583a875bbe5b1414997b949c0caf5dd5e: Status 404 returned error can't find the container with id 1e433bfa4a022e6f455754dae4fe1c4583a875bbe5b1414997b949c0caf5dd5e Jul 06 21:45:13.744553 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:13.744498 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" event={"ID":"e391781d-e9ed-4f8a-8a36-72e3526f2914","Type":"ContainerStarted","Data":"af24384e13d23dc33efd7f70f4010057fb03eb79d4e975003e77020bd712d57f"} Jul 06 21:45:13.744986 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:13.744665 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" Jul 06 21:45:13.744986 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:13.744707 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" event={"ID":"e391781d-e9ed-4f8a-8a36-72e3526f2914","Type":"ContainerStarted","Data":"1e433bfa4a022e6f455754dae4fe1c4583a875bbe5b1414997b949c0caf5dd5e"} Jul 06 21:45:13.761482 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:13.761431 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" podStartSLOduration=1.761417068 podStartE2EDuration="1.761417068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:45:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:45:13.760278571 +0000 UTC m=+1937.885259998" watchObservedRunningTime="2026-07-06 21:45:13.761417068 +0000 UTC m=+1937.886398490" Jul 06 21:45:13.843306 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:13.843273 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:45:19.753543 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:19.753514 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" Jul 06 21:45:55.948149 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:45:55.948072 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:21.184589 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:47:21.184514 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:56.445399 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:47:56.445369 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:47:56.447878 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:47:56.447857 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:47:56.448882 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:47:56.448863 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:47:56.451240 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:47:56.451219 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:49:09.275675 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:49:09.275629 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:09.334127 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:49:09.334093 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:56.470046 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:52:56.470011 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:52:56.473413 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:52:56.473391 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:52:56.473569 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:52:56.473394 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:52:56.476781 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:52:56.476764 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:53:27.101477 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:27.101379 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j"] Jul 06 21:53:27.102040 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:27.101760 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" podUID="e391781d-e9ed-4f8a-8a36-72e3526f2914" containerName="splitter-graph-563cb" containerID="cri-o://af24384e13d23dc33efd7f70f4010057fb03eb79d4e975003e77020bd712d57f" gracePeriod=30 Jul 06 21:53:27.151113 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:27.151080 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:28.814945 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:28.814905 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:29.752921 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:29.752879 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" podUID="e391781d-e9ed-4f8a-8a36-72e3526f2914" containerName="splitter-graph-563cb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:53:34.752391 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:34.752350 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" podUID="e391781d-e9ed-4f8a-8a36-72e3526f2914" containerName="splitter-graph-563cb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:53:39.751896 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:39.751851 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" podUID="e391781d-e9ed-4f8a-8a36-72e3526f2914" containerName="splitter-graph-563cb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:53:39.752390 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:39.751978 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" Jul 06 21:53:44.751991 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:44.751954 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" podUID="e391781d-e9ed-4f8a-8a36-72e3526f2914" containerName="splitter-graph-563cb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:53:49.753042 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:49.753002 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" podUID="e391781d-e9ed-4f8a-8a36-72e3526f2914" containerName="splitter-graph-563cb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:53:54.752531 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:54.752486 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" podUID="e391781d-e9ed-4f8a-8a36-72e3526f2914" containerName="splitter-graph-563cb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 21:53:57.751052 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.751028 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" Jul 06 21:53:57.829510 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.829456 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e391781d-e9ed-4f8a-8a36-72e3526f2914-openshift-service-ca-bundle\") pod \"e391781d-e9ed-4f8a-8a36-72e3526f2914\" (UID: \"e391781d-e9ed-4f8a-8a36-72e3526f2914\") " Jul 06 21:53:57.829694 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.829545 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e391781d-e9ed-4f8a-8a36-72e3526f2914-proxy-tls\") pod \"e391781d-e9ed-4f8a-8a36-72e3526f2914\" (UID: \"e391781d-e9ed-4f8a-8a36-72e3526f2914\") " Jul 06 21:53:57.829829 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.829803 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e391781d-e9ed-4f8a-8a36-72e3526f2914-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "e391781d-e9ed-4f8a-8a36-72e3526f2914" (UID: "e391781d-e9ed-4f8a-8a36-72e3526f2914"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:53:57.831516 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.831495 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e391781d-e9ed-4f8a-8a36-72e3526f2914-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e391781d-e9ed-4f8a-8a36-72e3526f2914" (UID: "e391781d-e9ed-4f8a-8a36-72e3526f2914"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:53:57.931018 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.930931 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e391781d-e9ed-4f8a-8a36-72e3526f2914-openshift-service-ca-bundle\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:53:57.931018 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.930964 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e391781d-e9ed-4f8a-8a36-72e3526f2914-proxy-tls\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 21:53:57.970049 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.970016 2577 generic.go:356] "Generic (PLEG): container finished" podID="e391781d-e9ed-4f8a-8a36-72e3526f2914" containerID="af24384e13d23dc33efd7f70f4010057fb03eb79d4e975003e77020bd712d57f" exitCode=0 Jul 06 21:53:57.970178 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.970057 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" Jul 06 21:53:57.970178 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.970095 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" event={"ID":"e391781d-e9ed-4f8a-8a36-72e3526f2914","Type":"ContainerDied","Data":"af24384e13d23dc33efd7f70f4010057fb03eb79d4e975003e77020bd712d57f"} Jul 06 21:53:57.970178 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.970147 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j" event={"ID":"e391781d-e9ed-4f8a-8a36-72e3526f2914","Type":"ContainerDied","Data":"1e433bfa4a022e6f455754dae4fe1c4583a875bbe5b1414997b949c0caf5dd5e"} Jul 06 21:53:57.970178 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.970169 2577 scope.go:117] "RemoveContainer" containerID="af24384e13d23dc33efd7f70f4010057fb03eb79d4e975003e77020bd712d57f" Jul 06 21:53:57.979091 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.979073 2577 scope.go:117] "RemoveContainer" containerID="af24384e13d23dc33efd7f70f4010057fb03eb79d4e975003e77020bd712d57f" Jul 06 21:53:57.979360 ip-10-0-136-12 kubenswrapper[2577]: E0706 21:53:57.979343 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af24384e13d23dc33efd7f70f4010057fb03eb79d4e975003e77020bd712d57f\": container with ID starting with af24384e13d23dc33efd7f70f4010057fb03eb79d4e975003e77020bd712d57f not found: ID does not exist" containerID="af24384e13d23dc33efd7f70f4010057fb03eb79d4e975003e77020bd712d57f" Jul 06 21:53:57.979507 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.979367 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af24384e13d23dc33efd7f70f4010057fb03eb79d4e975003e77020bd712d57f"} err="failed to get container status \"af24384e13d23dc33efd7f70f4010057fb03eb79d4e975003e77020bd712d57f\": rpc error: code = NotFound desc = could not find container \"af24384e13d23dc33efd7f70f4010057fb03eb79d4e975003e77020bd712d57f\": container with ID starting with af24384e13d23dc33efd7f70f4010057fb03eb79d4e975003e77020bd712d57f not found: ID does not exist" Jul 06 21:53:57.992974 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.992939 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j"] Jul 06 21:53:57.994882 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:57.994857 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-563cb-656565cb55-rbk2j"] Jul 06 21:53:58.490181 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:53:58.490144 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e391781d-e9ed-4f8a-8a36-72e3526f2914" path="/var/lib/kubelet/pods/e391781d-e9ed-4f8a-8a36-72e3526f2914/volumes" Jul 06 21:54:11.613809 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:54:11.613757 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:35.704011 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:54:35.703961 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 21:57:56.495394 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:57:56.495368 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:57:56.498932 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:57:56.498909 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:57:56.499790 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:57:56.499768 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 21:57:56.503452 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:57:56.503437 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 21:59:42.232810 ip-10-0-136-12 kubenswrapper[2577]: I0706 21:59:42.232724 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:21.361261 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:21.361221 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj"] Jul 06 22:01:21.362050 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:21.361515 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" podUID="8fb96fa6-d5fa-40b2-b61e-c079cf778c50" containerName="switch-graph-da1f6" containerID="cri-o://a3a36a493cf15d7ea50adc5821ee3af2ce9e545b2aa4ea7b0708e8be427ba333" gracePeriod=30 Jul 06 22:01:21.420564 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:21.420525 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:21.539453 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:21.539407 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:21.888309 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:21.888277 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:23.123779 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:23.123745 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:23.250102 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:23.250071 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:24.709232 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:24.709189 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" podUID="8fb96fa6-d5fa-40b2-b61e-c079cf778c50" containerName="switch-graph-da1f6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 22:01:29.709590 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:29.709550 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" podUID="8fb96fa6-d5fa-40b2-b61e-c079cf778c50" containerName="switch-graph-da1f6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 22:01:34.709358 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:34.709315 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" podUID="8fb96fa6-d5fa-40b2-b61e-c079cf778c50" containerName="switch-graph-da1f6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 22:01:34.709783 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:34.709428 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 22:01:38.972182 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:38.972152 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:39.709204 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:39.709164 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" podUID="8fb96fa6-d5fa-40b2-b61e-c079cf778c50" containerName="switch-graph-da1f6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 22:01:39.781393 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:39.781362 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:40.608217 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:40.608190 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:41.391518 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:41.391486 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:42.195823 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:42.195795 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:43.022687 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:43.022659 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:43.803791 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:43.803762 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:44.641834 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:44.641803 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:44.709701 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:44.709662 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" podUID="8fb96fa6-d5fa-40b2-b61e-c079cf778c50" containerName="switch-graph-da1f6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 22:01:45.467236 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:45.467211 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:46.296991 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:46.296960 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:47.156497 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:47.156449 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:48.041898 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:48.041865 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:48.831937 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:48.831898 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:49.615376 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:49.615347 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-da1f6-ccc959c7-b2xwj_8fb96fa6-d5fa-40b2-b61e-c079cf778c50/switch-graph-da1f6/0.log" Jul 06 22:01:49.709325 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:49.709280 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" podUID="8fb96fa6-d5fa-40b2-b61e-c079cf778c50" containerName="switch-graph-da1f6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 22:01:51.496768 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.496742 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 22:01:51.580231 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.580202 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-openshift-service-ca-bundle\") pod \"8fb96fa6-d5fa-40b2-b61e-c079cf778c50\" (UID: \"8fb96fa6-d5fa-40b2-b61e-c079cf778c50\") " Jul 06 22:01:51.580441 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.580304 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-proxy-tls\") pod \"8fb96fa6-d5fa-40b2-b61e-c079cf778c50\" (UID: \"8fb96fa6-d5fa-40b2-b61e-c079cf778c50\") " Jul 06 22:01:51.580597 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.580576 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "8fb96fa6-d5fa-40b2-b61e-c079cf778c50" (UID: "8fb96fa6-d5fa-40b2-b61e-c079cf778c50"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:01:51.582441 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.582415 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8fb96fa6-d5fa-40b2-b61e-c079cf778c50" (UID: "8fb96fa6-d5fa-40b2-b61e-c079cf778c50"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:01:51.681345 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.681264 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-openshift-service-ca-bundle\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 22:01:51.681345 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.681292 2577 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8fb96fa6-d5fa-40b2-b61e-c079cf778c50-proxy-tls\") on node \"ip-10-0-136-12.ec2.internal\" DevicePath \"\"" Jul 06 22:01:51.742396 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.742333 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:51.905783 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.905745 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:51.964955 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.964852 2577 generic.go:356] "Generic (PLEG): container finished" podID="8fb96fa6-d5fa-40b2-b61e-c079cf778c50" containerID="a3a36a493cf15d7ea50adc5821ee3af2ce9e545b2aa4ea7b0708e8be427ba333" exitCode=0 Jul 06 22:01:51.964955 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.964893 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" Jul 06 22:01:51.965198 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.964947 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" event={"ID":"8fb96fa6-d5fa-40b2-b61e-c079cf778c50","Type":"ContainerDied","Data":"a3a36a493cf15d7ea50adc5821ee3af2ce9e545b2aa4ea7b0708e8be427ba333"} Jul 06 22:01:51.965198 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.965000 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj" event={"ID":"8fb96fa6-d5fa-40b2-b61e-c079cf778c50","Type":"ContainerDied","Data":"6eacf5a27bb67b7badad16aa1adc783834e4ad99268f188b11a6f614ed11347a"} Jul 06 22:01:51.965198 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.965033 2577 scope.go:117] "RemoveContainer" containerID="a3a36a493cf15d7ea50adc5821ee3af2ce9e545b2aa4ea7b0708e8be427ba333" Jul 06 22:01:51.965430 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.965323 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:51.973822 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.973805 2577 scope.go:117] "RemoveContainer" containerID="a3a36a493cf15d7ea50adc5821ee3af2ce9e545b2aa4ea7b0708e8be427ba333" Jul 06 22:01:51.974082 ip-10-0-136-12 kubenswrapper[2577]: E0706 22:01:51.974063 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3a36a493cf15d7ea50adc5821ee3af2ce9e545b2aa4ea7b0708e8be427ba333\": container with ID starting with a3a36a493cf15d7ea50adc5821ee3af2ce9e545b2aa4ea7b0708e8be427ba333 not found: ID does not exist" containerID="a3a36a493cf15d7ea50adc5821ee3af2ce9e545b2aa4ea7b0708e8be427ba333" Jul 06 22:01:51.974143 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.974097 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3a36a493cf15d7ea50adc5821ee3af2ce9e545b2aa4ea7b0708e8be427ba333"} err="failed to get container status \"a3a36a493cf15d7ea50adc5821ee3af2ce9e545b2aa4ea7b0708e8be427ba333\": rpc error: code = NotFound desc = could not find container \"a3a36a493cf15d7ea50adc5821ee3af2ce9e545b2aa4ea7b0708e8be427ba333\": container with ID starting with a3a36a493cf15d7ea50adc5821ee3af2ce9e545b2aa4ea7b0708e8be427ba333 not found: ID does not exist" Jul 06 22:01:51.984939 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.984908 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj"] Jul 06 22:01:51.990881 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:51.990857 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-da1f6-ccc959c7-b2xwj"] Jul 06 22:01:52.492212 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:52.492176 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fb96fa6-d5fa-40b2-b61e-c079cf778c50" path="/var/lib/kubelet/pods/8fb96fa6-d5fa-40b2-b61e-c079cf778c50/volumes" Jul 06 22:01:54.491125 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:54.491077 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xjshg_c8194ca3-5e42-4623-aabb-091c40445eaa/global-pull-secret-syncer/0.log" Jul 06 22:01:54.578748 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:54.578721 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" Jul 06 22:01:54.684813 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:54.684783 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-12.ec2.internal_54f38d8f384b65c8b8874f57c20a13d4/haproxy/0.log" Jul 06 22:01:57.772395 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:57.772343 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:57.873692 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:57.873642 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:59.403942 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:59.403899 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-c6wd6_f922a864-b75a-4e41-afa7-d868357ceff9/cluster-monitoring-operator/0.log" Jul 06 22:01:59.429339 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:59.429310 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-gzzch_a25e5d09-1c8a-42b2-9cc5-edbf059149df/kube-state-metrics/0.log" Jul 06 22:01:59.453447 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:59.453426 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-gzzch_a25e5d09-1c8a-42b2-9cc5-edbf059149df/kube-rbac-proxy-main/0.log" Jul 06 22:01:59.478348 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:59.478322 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-gzzch_a25e5d09-1c8a-42b2-9cc5-edbf059149df/kube-rbac-proxy-self/0.log" Jul 06 22:01:59.653111 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:59.653082 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dxwml_0775089d-2c9e-40ae-81c1-503952781d10/node-exporter/0.log" Jul 06 22:01:59.687295 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:59.687224 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dxwml_0775089d-2c9e-40ae-81c1-503952781d10/kube-rbac-proxy/0.log" Jul 06 22:01:59.709708 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:59.709678 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dxwml_0775089d-2c9e-40ae-81c1-503952781d10/init-textfile/0.log" Jul 06 22:01:59.808368 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:59.808342 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-tpzgp_2133331e-43f5-4a7b-a292-820b9469769c/kube-rbac-proxy-main/0.log" Jul 06 22:01:59.828281 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:59.828239 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-tpzgp_2133331e-43f5-4a7b-a292-820b9469769c/kube-rbac-proxy-self/0.log" Jul 06 22:01:59.848081 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:01:59.848062 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-tpzgp_2133331e-43f5-4a7b-a292-820b9469769c/openshift-state-metrics/0.log" Jul 06 22:02:00.134658 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:00.134608 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-2ntkj_0762a8e1-6ebc-48c2-bbf9-07cdd20d480a/prometheus-operator-admission-webhook/0.log" Jul 06 22:02:01.800167 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:01.800106 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ddkph_2fc778ca-01e3-41a8-87ae-9e5d28afcaad/konnectivity-agent/0.log" limit=1024 Jul 06 22:02:01.957454 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:01.957425 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/1.log" Jul 06 22:02:01.962303 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:01.962275 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-p4mff_5b436268-66a4-40f6-9386-1d0ee9b180da/console-operator/2.log" Jul 06 22:02:02.813701 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.813668 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s"] Jul 06 22:02:02.814073 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.813979 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fb96fa6-d5fa-40b2-b61e-c079cf778c50" containerName="switch-graph-da1f6" Jul 06 22:02:02.814073 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.813990 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fb96fa6-d5fa-40b2-b61e-c079cf778c50" containerName="switch-graph-da1f6" Jul 06 22:02:02.814073 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.814001 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e391781d-e9ed-4f8a-8a36-72e3526f2914" containerName="splitter-graph-563cb" Jul 06 22:02:02.814073 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.814007 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e391781d-e9ed-4f8a-8a36-72e3526f2914" containerName="splitter-graph-563cb" Jul 06 22:02:02.814073 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.814049 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e391781d-e9ed-4f8a-8a36-72e3526f2914" containerName="splitter-graph-563cb" Jul 06 22:02:02.814073 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.814061 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fb96fa6-d5fa-40b2-b61e-c079cf778c50" containerName="switch-graph-da1f6" Jul 06 22:02:02.818335 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.818316 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:02.820871 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.820850 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-d2pl7\"/\"default-dockercfg-hwbkv\"" Jul 06 22:02:02.821646 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.821630 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d2pl7\"/\"openshift-service-ca.crt\"" Jul 06 22:02:02.821700 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.821662 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d2pl7\"/\"kube-root-ca.crt\"" Jul 06 22:02:02.828160 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.828138 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s"] Jul 06 22:02:02.970830 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.970803 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mxck\" (UniqueName: \"kubernetes.io/projected/96a45009-13f0-4b52-bebf-681c7fdbc0d5-kube-api-access-6mxck\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:02.970830 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.970834 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96a45009-13f0-4b52-bebf-681c7fdbc0d5-sys\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:02.971040 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.970860 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/96a45009-13f0-4b52-bebf-681c7fdbc0d5-proc\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:02.971040 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.970902 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96a45009-13f0-4b52-bebf-681c7fdbc0d5-lib-modules\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:02.971040 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:02.970961 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/96a45009-13f0-4b52-bebf-681c7fdbc0d5-podres\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:03.072279 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:03.072251 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/96a45009-13f0-4b52-bebf-681c7fdbc0d5-podres\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:03.072429 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:03.072314 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mxck\" (UniqueName: \"kubernetes.io/projected/96a45009-13f0-4b52-bebf-681c7fdbc0d5-kube-api-access-6mxck\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:03.072429 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:03.072341 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96a45009-13f0-4b52-bebf-681c7fdbc0d5-sys\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:03.072429 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:03.072373 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/96a45009-13f0-4b52-bebf-681c7fdbc0d5-proc\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:03.072429 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:03.072393 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96a45009-13f0-4b52-bebf-681c7fdbc0d5-lib-modules\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:03.072594 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:03.072438 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/96a45009-13f0-4b52-bebf-681c7fdbc0d5-podres\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:03.072594 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:03.072484 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/96a45009-13f0-4b52-bebf-681c7fdbc0d5-proc\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:03.072594 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:03.072487 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96a45009-13f0-4b52-bebf-681c7fdbc0d5-sys\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:03.072594 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:03.072532 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96a45009-13f0-4b52-bebf-681c7fdbc0d5-lib-modules\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:03.081647 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:03.081628 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mxck\" (UniqueName: \"kubernetes.io/projected/96a45009-13f0-4b52-bebf-681c7fdbc0d5-kube-api-access-6mxck\") pod \"perf-node-gather-daemonset-vtx7s\" (UID: \"96a45009-13f0-4b52-bebf-681c7fdbc0d5\") " pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:03.128671 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:03.128639 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:03.257324 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:03.257192 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s"] Jul 06 22:02:03.259510 ip-10-0-136-12 kubenswrapper[2577]: W0706 22:02:03.259484 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod96a45009_13f0_4b52_bebf_681c7fdbc0d5.slice/crio-e672a5276ec6b2b360689982ecca897757097030da690ed4531098d438ec6d14 WatchSource:0}: Error finding container e672a5276ec6b2b360689982ecca897757097030da690ed4531098d438ec6d14: Status 404 returned error can't find the container with id e672a5276ec6b2b360689982ecca897757097030da690ed4531098d438ec6d14 Jul 06 22:02:04.013474 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:04.013432 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w8w2r_de552445-923f-48e7-9a47-60f066827b50/dns/0.log" Jul 06 22:02:04.022622 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:04.022593 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" event={"ID":"96a45009-13f0-4b52-bebf-681c7fdbc0d5","Type":"ContainerStarted","Data":"af3c49ec83b018eabbc6d1b98ebae97a123ff7b9096afd72d26f207862690751"} Jul 06 22:02:04.022763 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:04.022646 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" event={"ID":"96a45009-13f0-4b52-bebf-681c7fdbc0d5","Type":"ContainerStarted","Data":"e672a5276ec6b2b360689982ecca897757097030da690ed4531098d438ec6d14"} Jul 06 22:02:04.022763 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:04.022665 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:04.041364 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:04.041320 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" podStartSLOduration=2.041306636 podStartE2EDuration="2.041306636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:02:04.040956476 +0000 UTC m=+2948.165937900" watchObservedRunningTime="2026-07-06 22:02:04.041306636 +0000 UTC m=+2948.166288059" Jul 06 22:02:04.043182 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:04.043164 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w8w2r_de552445-923f-48e7-9a47-60f066827b50/kube-rbac-proxy/0.log" Jul 06 22:02:04.114638 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:04.114609 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r9fcz_6a1be0cb-488e-4e55-855e-06abda43369b/dns-node-resolver/0.log" Jul 06 22:02:04.626455 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:04.626392 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8w488_b490d547-2ba8-44f0-a6e6-898846f59cc9/node-ca/0.log" Jul 06 22:02:05.783145 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:05.783119 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dwg5d_78439b85-9901-45a4-898d-dbeb91d2eacc/serve-healthcheck-canary/0.log" Jul 06 22:02:06.315505 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:06.315445 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lsz7j_f1e609dd-7208-45c8-93fb-019ef2780f55/kube-rbac-proxy/0.log" Jul 06 22:02:06.337826 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:06.337803 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lsz7j_f1e609dd-7208-45c8-93fb-019ef2780f55/exporter/0.log" Jul 06 22:02:06.357861 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:06.357829 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lsz7j_f1e609dd-7208-45c8-93fb-019ef2780f55/extractor/0.log" Jul 06 22:02:08.351850 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:08.351816 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-7ccfc88547-nqxz9_bfff6677-9f60-42a3-a03e-9dbfdf63e7d3/manager/0.log" Jul 06 22:02:08.370970 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:08.370943 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-67d95fdc48-lvd4m_a45f6536-6c65-4922-ba70-19fc52da1d94/manager/0.log" Jul 06 22:02:10.036943 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:10.036917 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-d2pl7/perf-node-gather-daemonset-vtx7s" Jul 06 22:02:12.637593 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:12.637565 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-9x8sj_f40d04cd-143d-44b0-9367-416c23b8d895/migrator/0.log" Jul 06 22:02:12.657921 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:12.657891 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-9x8sj_f40d04cd-143d-44b0-9367-416c23b8d895/graceful-termination/0.log" Jul 06 22:02:13.847739 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:13.847708 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7ds6m_d291c60c-4d03-4c4d-8284-0f3d918d5ad1/kube-multus-additional-cni-plugins/0.log" Jul 06 22:02:13.868390 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:13.868366 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7ds6m_d291c60c-4d03-4c4d-8284-0f3d918d5ad1/egress-router-binary-copy/0.log" Jul 06 22:02:13.888644 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:13.888622 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7ds6m_d291c60c-4d03-4c4d-8284-0f3d918d5ad1/cni-plugins/0.log" Jul 06 22:02:13.907516 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:13.907493 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7ds6m_d291c60c-4d03-4c4d-8284-0f3d918d5ad1/bond-cni-plugin/0.log" Jul 06 22:02:13.928387 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:13.928363 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7ds6m_d291c60c-4d03-4c4d-8284-0f3d918d5ad1/routeoverride-cni/0.log" Jul 06 22:02:13.949002 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:13.948981 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7ds6m_d291c60c-4d03-4c4d-8284-0f3d918d5ad1/whereabouts-cni-bincopy/0.log" Jul 06 22:02:13.967900 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:13.967879 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7ds6m_d291c60c-4d03-4c4d-8284-0f3d918d5ad1/whereabouts-cni/0.log" Jul 06 22:02:14.296742 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:14.296711 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-h4886_b39d0a10-298e-4d2b-8469-0194658c0466/kube-multus/0.log" Jul 06 22:02:14.457054 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:14.456969 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xv8m9_54c01a9d-5eb8-4b7d-959a-f2a6e6a43150/network-metrics-daemon/0.log" Jul 06 22:02:14.474721 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:14.474696 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xv8m9_54c01a9d-5eb8-4b7d-959a-f2a6e6a43150/kube-rbac-proxy/0.log" Jul 06 22:02:15.380817 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:15.380785 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-controller/0.log" Jul 06 22:02:15.398421 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:15.398393 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/1.log" Jul 06 22:02:15.398596 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:15.398524 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovn-acl-logging/0.log" Jul 06 22:02:15.417880 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:15.417859 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/kube-rbac-proxy-node/0.log" Jul 06 22:02:15.439520 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:15.439490 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/kube-rbac-proxy-ovn-metrics/0.log" Jul 06 22:02:15.457609 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:15.457585 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/northd/0.log" Jul 06 22:02:15.477564 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:15.477544 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/nbdb/0.log" Jul 06 22:02:15.498067 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:15.498045 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/sbdb/0.log" Jul 06 22:02:15.538417 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:15.538374 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g64lt_c6690861-f325-41b3-bf8c-5e43798ec3d0/ovnkube-controller/0.log" Jul 06 22:02:16.759066 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:16.759039 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-mzmmk_b821193f-628f-4ed3-b87d-e96f3c441ff7/network-check-target-container/0.log" Jul 06 22:02:17.627763 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:17.627730 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-98smj_6753e71a-7591-4dca-b1ba-236c3e51c791/iptables-alerter/0.log" Jul 06 22:02:18.274298 ip-10-0-136-12 kubenswrapper[2577]: I0706 22:02:18.274270 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-crh49_701babd3-9d0a-462d-b559-7508529a25f4/tuned/0.log"