Jul 16 00:13:10.809844 ip-10-0-128-224 systemd[1]: Starting Kubernetes Kubelet... Jul 16 00:13:11.209372 ip-10-0-128-224 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 00:13:11.209372 ip-10-0-128-224 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 00:13:11.209372 ip-10-0-128-224 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 00:13:11.209372 ip-10-0-128-224 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 00:13:11.209372 ip-10-0-128-224 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 00:13:11.211016 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.210922 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 00:13:11.215258 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215237 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 00:13:11.215258 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215253 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 00:13:11.215258 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215258 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 00:13:11.215258 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215262 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 00:13:11.215258 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215266 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215271 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215275 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215280 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215284 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215288 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215291 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215295 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215317 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215321 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215325 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215329 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215332 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215336 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215344 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215350 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215355 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215359 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215363 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215367 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 00:13:11.215557 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215371 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215375 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215379 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215383 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215386 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215391 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215395 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215400 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215404 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215408 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215412 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215416 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215420 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215424 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215429 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215434 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215441 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215447 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215452 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 00:13:11.216364 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215456 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215461 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215466 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215470 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215475 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215479 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215484 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215488 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215492 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215496 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215501 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215505 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215509 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215514 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215518 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215522 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215526 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215544 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215549 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215556 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 00:13:11.217178 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215560 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215564 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215568 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215574 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215578 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215583 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215586 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215592 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215599 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215604 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215608 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215613 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215617 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215621 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215626 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215632 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215638 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215643 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215647 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215651 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 00:13:11.218007 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215656 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215660 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215664 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215669 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215679 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215684 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215688 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215692 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215697 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215702 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215707 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215711 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215717 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215721 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215725 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.215729 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217061 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217073 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217079 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217085 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 00:13:11.218851 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217089 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217094 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217098 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217102 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217107 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217111 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217115 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217120 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217124 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217131 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217138 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217143 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217148 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217153 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217160 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217166 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217171 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217175 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217179 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 00:13:11.219738 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217184 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217188 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217193 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217197 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217202 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217206 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217212 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217217 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217221 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217227 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217231 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217235 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217240 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217244 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217248 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217252 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217257 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217262 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217266 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217270 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 00:13:11.220523 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217275 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217279 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217283 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217287 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217292 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217296 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217299 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217304 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217309 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217313 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217318 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217322 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217326 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217331 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217335 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217340 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217344 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217348 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217352 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217358 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 00:13:11.221141 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217362 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217366 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217370 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217374 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217379 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217384 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217388 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217392 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217396 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217401 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217405 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217409 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217413 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217417 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217421 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217426 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217429 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217433 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217437 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217441 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 00:13:11.221767 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217445 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217449 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217454 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217458 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217465 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217469 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217476 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217482 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217487 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217492 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217496 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217501 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217507 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217512 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217516 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.217521 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218200 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218209 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218215 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 00:13:11.222239 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218219 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218224 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218228 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218232 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218237 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218241 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218246 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218250 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218254 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218259 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218263 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218267 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218271 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218275 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218281 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218287 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218292 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218296 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218300 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 00:13:11.222809 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218305 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218309 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218313 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218318 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218322 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218327 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218332 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218337 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218341 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218345 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218349 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218354 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218359 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218364 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218368 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218372 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218376 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218380 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218385 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218389 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 00:13:11.223261 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218395 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218399 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218403 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218408 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218412 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218416 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218420 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218424 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218428 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218433 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218437 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218441 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218448 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218455 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218460 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218464 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218469 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218474 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218479 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 00:13:11.223780 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218488 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218492 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218496 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218502 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218506 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218510 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218515 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218519 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218523 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218527 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218548 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218552 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218557 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218561 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218565 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218569 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218574 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218578 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218582 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218586 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 00:13:11.224242 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218590 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218594 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218601 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218606 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218611 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218619 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218623 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218628 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218633 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218639 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218643 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218648 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218652 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218658 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218662 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218666 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218670 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.218674 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218778 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218789 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 00:13:11.224745 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218798 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218805 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218812 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218817 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218824 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218830 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218838 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218843 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218848 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218853 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218858 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218862 2564 flags.go:64] FLAG: --cgroup-root="" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218867 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218871 2564 flags.go:64] FLAG: --client-ca-file="" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218876 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218880 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218889 2564 flags.go:64] FLAG: --cluster-domain="" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218895 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218901 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218906 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218911 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218917 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218922 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 00:13:11.225224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218927 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218933 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218937 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218943 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218947 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218952 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218957 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218964 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218969 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218974 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218979 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218984 2564 flags.go:64] FLAG: --enable-server="true" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218989 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.218998 2564 flags.go:64] FLAG: --event-burst="100" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219004 2564 flags.go:64] FLAG: --event-qps="50" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219009 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219014 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219019 2564 flags.go:64] FLAG: --eviction-hard="" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219025 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219030 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219035 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219040 2564 flags.go:64] FLAG: --eviction-soft="" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219044 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219049 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219053 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 00:13:11.225868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219058 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219065 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219069 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219074 2564 flags.go:64] FLAG: --feature-gates="" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219080 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219084 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219089 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219094 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219100 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219104 2564 flags.go:64] FLAG: --help="false" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219109 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219115 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219120 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219125 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219130 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219136 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219140 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219145 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219150 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219154 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219159 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219164 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219170 2564 flags.go:64] FLAG: --kube-reserved="" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219175 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 00:13:11.226520 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219179 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219184 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219189 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219194 2564 flags.go:64] FLAG: --lock-file="" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219198 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219203 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219208 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219217 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219223 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219229 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219234 2564 flags.go:64] FLAG: --logging-format="text" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219238 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219244 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219248 2564 flags.go:64] FLAG: --manifest-url="" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219253 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219260 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219265 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219272 2564 flags.go:64] FLAG: --max-pods="110" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219277 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219282 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219288 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219292 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219297 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219302 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219307 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 00:13:11.227255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219318 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219323 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219328 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219333 2564 flags.go:64] FLAG: --pod-cidr="" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219337 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219347 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219351 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219356 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219361 2564 flags.go:64] FLAG: --port="10250" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219366 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219371 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-085ff36c7a26a2dbe" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219376 2564 flags.go:64] FLAG: --qos-reserved="" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219381 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219386 2564 flags.go:64] FLAG: --register-node="true" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219391 2564 flags.go:64] FLAG: --register-with-taints="" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219397 2564 flags.go:64] FLAG: --registry-burst="10" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219401 2564 flags.go:64] FLAG: --registry-qps="5" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219407 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219412 2564 flags.go:64] FLAG: --reserved-memory="" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219419 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219424 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219429 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219433 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219438 2564 flags.go:64] FLAG: --runonce="false" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219442 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 00:13:11.228142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219447 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219452 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219457 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219462 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219467 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219472 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219477 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219481 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219486 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219490 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219495 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219500 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219505 2564 flags.go:64] FLAG: --system-cgroups="" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219509 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219518 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219523 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219528 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219552 2564 flags.go:64] FLAG: --tls-min-version="" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219557 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219562 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219567 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219572 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219576 2564 flags.go:64] FLAG: --v="2" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219583 2564 flags.go:64] FLAG: --version="false" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219591 2564 flags.go:64] FLAG: --vmodule="" Jul 16 00:13:11.228797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219597 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.219603 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219793 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219800 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219805 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219810 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219815 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219819 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219823 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219827 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219832 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219838 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219845 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219850 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219855 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219859 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219864 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219869 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219873 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219879 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 00:13:11.229392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219883 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219887 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219892 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219897 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219901 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219905 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219910 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219914 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219918 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219922 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219927 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219933 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219937 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219941 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219945 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219949 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219953 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219957 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219961 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 00:13:11.229892 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219965 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219969 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219973 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219978 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219982 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219987 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219991 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.219995 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220000 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220003 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220007 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220012 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220017 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220021 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220025 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220030 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220033 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220037 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220042 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220047 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 00:13:11.230392 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220051 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220055 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220059 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220063 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220069 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220073 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220077 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220081 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220085 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220089 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220094 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220098 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220102 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220106 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220110 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220115 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220119 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220123 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220128 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220131 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 00:13:11.230998 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220136 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220140 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220145 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220150 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220163 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220168 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220173 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220177 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220182 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220186 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220191 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220197 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220201 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220205 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220210 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220215 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220222 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220226 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220230 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 00:13:11.231478 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220236 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220242 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.220246 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.220255 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.228709 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.228727 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.228747 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.228753 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228815 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228821 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228824 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228828 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228830 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228834 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228836 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 00:13:11.231967 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228839 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228842 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228845 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228847 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228850 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228852 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228855 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228857 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228861 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228864 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228866 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228869 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228872 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228874 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228877 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228879 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228882 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228885 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228887 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228890 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 00:13:11.232352 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228892 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228895 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228900 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228904 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228906 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228909 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228912 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228916 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228918 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228921 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228923 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228926 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228929 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228931 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228934 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228936 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228939 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228941 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228944 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228948 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 00:13:11.232896 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228951 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228953 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228956 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228958 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228961 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228963 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228966 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228968 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228971 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228974 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228976 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228979 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228982 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228985 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228990 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228993 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228996 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.228999 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229002 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229005 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 00:13:11.233375 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229008 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229011 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229013 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229016 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229018 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229021 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229024 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229026 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229029 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229031 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229034 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229038 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229042 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229045 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229048 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229051 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229054 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229057 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229059 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 00:13:11.233869 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229062 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229065 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229067 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229070 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229073 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229075 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229077 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229080 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229083 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229085 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229088 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229091 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229094 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.229099 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229219 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 00:13:11.234333 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229224 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229227 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229230 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229233 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229236 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229238 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229241 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229243 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229246 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229249 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229251 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229255 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229257 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229260 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229263 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229265 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229268 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229271 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229273 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 00:13:11.234725 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229276 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229279 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229283 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229285 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229288 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229290 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229293 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229296 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229299 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229301 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229304 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229308 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229311 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229314 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229317 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229320 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229323 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229326 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229329 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 00:13:11.235187 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229331 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229334 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229336 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229339 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229342 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229344 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229347 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229349 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229352 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229354 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229357 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229359 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229362 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229365 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229367 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229370 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229372 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229375 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229378 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229381 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 00:13:11.235804 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229383 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229385 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229388 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229390 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229393 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229395 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229398 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229402 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229406 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229409 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229411 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229414 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229417 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229419 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229422 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229424 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229427 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229429 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229432 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 00:13:11.236285 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229435 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229437 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229440 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229442 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229445 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229448 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229450 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229453 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229455 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229458 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229460 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229463 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229466 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229469 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229471 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229474 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229476 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229479 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229481 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229484 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 00:13:11.236770 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:11.229486 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 00:13:11.237247 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.229491 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 00:13:11.237247 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.229635 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 00:13:11.237247 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.231916 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 00:13:11.237247 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.233151 2564 server.go:1019] "Starting client certificate rotation" Jul 16 00:13:11.237247 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.233256 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 00:13:11.237247 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.233326 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 00:13:11.237247 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.235400 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 00:13:11.238368 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.238353 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 00:13:11.247718 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.247696 2564 log.go:25] "Validated CRI v1 runtime API" Jul 16 00:13:11.250044 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.250026 2564 log.go:25] "Validated CRI v1 image API" Jul 16 00:13:11.251462 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.251446 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 00:13:11.255234 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.255209 2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a5ada85d-a187-4c84-91b3-d4a43aa61042:/dev/nvme0n1p3 fe8980e4-c02b-4607-84d4-15d564652bd2:/dev/nvme0n1p4] Jul 16 00:13:11.255295 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.255235 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 00:13:11.259583 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.259566 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 00:13:11.261423 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.261088 2564 manager.go:216] Machine: {Timestamp:2026-07-16 00:13:11.258780387 +0000 UTC m=+0.326720853 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098769 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec276aadd4bcf4fceedc678e68700d24 SystemUUID:ec276aad-d4bc-f4fc-eedc-678e68700d24 BootID:af1d16c2-5b6e-44fd-b22c-895b1d6a7c62 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:19:3b:44:fe:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:19:3b:44:fe:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:ad:1c:36:05:a5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 00:13:11.261423 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.261298 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 00:13:11.261621 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.261510 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 00:13:11.265524 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.265492 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 00:13:11.265702 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.265526 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-224.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 00:13:11.266259 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.266247 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 00:13:11.266312 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.266264 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 00:13:11.266312 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.266274 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 00:13:11.267628 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.267614 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 00:13:11.268891 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.268879 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 00:13:11.271712 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.271697 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 16 00:13:11.271852 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.271840 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 00:13:11.271906 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.271880 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 16 00:13:11.271906 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.271895 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 00:13:11.271993 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.271916 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 00:13:11.271993 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.271931 2564 kubelet.go:398] "Adding apiserver pod source" Jul 16 00:13:11.271993 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.271943 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 00:13:11.272337 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.272319 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 00:13:11.272337 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.272342 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 00:13:11.275001 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.274984 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 00:13:11.276785 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.276772 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 00:13:11.276839 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.276802 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 00:13:11.276987 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.276976 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 00:13:11.277022 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.276992 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 00:13:11.277022 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.276998 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 00:13:11.277022 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.277004 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 00:13:11.277022 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.277013 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 00:13:11.277022 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.277019 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 00:13:11.277466 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.277457 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 00:13:11.277466 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.277466 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 00:13:11.277528 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.277473 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 00:13:11.277528 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.277479 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 00:13:11.277528 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.277487 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 00:13:11.277528 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.277495 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 00:13:11.278809 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.278795 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 00:13:11.278846 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.278813 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 00:13:11.282185 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.282168 2564 server.go:1268] "Started kubelet" Jul 16 00:13:11.282276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.282246 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 00:13:11.282347 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.282253 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 00:13:11.282347 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.282337 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 00:13:11.282837 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.282817 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-224.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 00:13:11.283004 ip-10-0-128-224 systemd[1]: Started Kubernetes Kubelet. Jul 16 00:13:11.283156 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.283003 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 00:13:11.283156 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.283003 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-224.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 00:13:11.283461 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.283409 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 00:13:11.284088 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.284077 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 16 00:13:11.285448 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.285425 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x64fz" Jul 16 00:13:11.289441 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.289419 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 00:13:11.289666 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.289647 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 00:13:11.289753 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.289703 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 00:13:11.289753 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.289735 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:11.290224 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.288978 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-224.ec2.internal.18c29cc96a8d3b25 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-224.ec2.internal,UID:ip-10-0-128-224.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-224.ec2.internal,},FirstTimestamp:2026-07-16 00:13:11.282137893 +0000 UTC m=+0.350078359,LastTimestamp:2026-07-16 00:13:11.282137893 +0000 UTC m=+0.350078359,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-224.ec2.internal,}" Jul 16 00:13:11.290320 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.290262 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 00:13:11.290384 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.290372 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 00:13:11.290439 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.290386 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 00:13:11.290665 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.290644 2564 factory.go:55] Registering systemd factory Jul 16 00:13:11.290665 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.290665 2564 factory.go:223] Registration of the systemd container factory successfully Jul 16 00:13:11.290665 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.290653 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 16 00:13:11.290852 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.290675 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 16 00:13:11.290911 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.290898 2564 factory.go:153] Registering CRI-O factory Jul 16 00:13:11.290963 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.290916 2564 factory.go:223] Registration of the crio container factory successfully Jul 16 00:13:11.291008 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.290967 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 00:13:11.291008 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.291001 2564 factory.go:103] Registering Raw factory Jul 16 00:13:11.291106 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.291017 2564 manager.go:1196] Started watching for new ooms in manager Jul 16 00:13:11.291352 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.291338 2564 manager.go:349] Starting recovery of all containers Jul 16 00:13:11.293104 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.293082 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x64fz" Jul 16 00:13:11.295858 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.295826 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-224.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 00:13:11.295986 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.295947 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 00:13:11.302909 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.302891 2564 manager.go:354] Recovery completed Jul 16 00:13:11.307839 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.307824 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:13:11.309849 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.309831 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:13:11.309919 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.309862 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:13:11.309919 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.309875 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:13:11.310446 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.310430 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 00:13:11.310446 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.310443 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:13:11.310563 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.310444 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 00:13:11.310563 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.310502 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 16 00:13:11.311216 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.311202 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:13:11.311284 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.311229 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:13:11.311284 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.311238 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:13:11.311837 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.311826 2564 policy_none.go:49] "None policy: Start" Jul 16 00:13:11.311874 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.311842 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 00:13:11.311874 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.311852 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 00:13:11.312852 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.312841 2564 policy_none.go:47] "Start" Jul 16 00:13:11.358447 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.358231 2564 manager.go:337] "Starting Device Plugin manager" Jul 16 00:13:11.358592 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.358479 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 00:13:11.358592 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.358492 2564 server.go:87] "Starting device plugin registration server" Jul 16 00:13:11.358874 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.358861 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 00:13:11.358907 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.358877 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 00:13:11.359017 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.358989 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 00:13:11.359113 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.359092 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 00:13:11.359113 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.359103 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 00:13:11.359792 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.359772 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 00:13:11.359858 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.359838 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:11.460024 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.459936 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:13:11.461229 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.460951 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:13:11.461229 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.460978 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:13:11.461229 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.460988 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:13:11.461229 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.461018 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.468234 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.468212 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.468293 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.468242 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-224.ec2.internal\": node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:11.476928 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.476898 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 00:13:11.478165 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.478148 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 00:13:11.478300 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.478189 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 00:13:11.478300 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.478271 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 00:13:11.478385 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.478312 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 00:13:11.480317 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.480302 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 00:13:11.481989 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.481975 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:11.579023 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.578981 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-224.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal"] Jul 16 00:13:11.579174 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.579072 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:13:11.580396 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.580375 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:13:11.580526 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.580404 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:13:11.580526 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.580424 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:13:11.581720 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.581706 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:13:11.581862 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.581833 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.581921 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.581882 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:13:11.582017 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.582005 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:11.582437 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.582420 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:13:11.582501 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.582452 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:13:11.582501 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.582462 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:13:11.582631 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.582420 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:13:11.582631 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.582610 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:13:11.582631 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.582621 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:13:11.583747 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.583732 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.583813 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.583757 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 00:13:11.584444 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.584429 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientMemory" Jul 16 00:13:11.584506 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.584459 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 00:13:11.584506 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.584472 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeHasSufficientPID" Jul 16 00:13:11.592445 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.592426 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e9f64c09e4b0a625629b9a3434bf1892-config\") pod \"kube-apiserver-proxy-ip-10-0-128-224.ec2.internal\" (UID: \"e9f64c09e4b0a625629b9a3434bf1892\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.592521 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.592452 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/77dcdf2501699d64463907b44b284337-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal\" (UID: \"77dcdf2501699d64463907b44b284337\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.592521 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.592468 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77dcdf2501699d64463907b44b284337-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal\" (UID: \"77dcdf2501699d64463907b44b284337\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.606195 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.606174 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-224.ec2.internal\" not found" node="ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.610772 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.610755 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-224.ec2.internal\" not found" node="ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.682612 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.682582 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:11.692977 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.692947 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e9f64c09e4b0a625629b9a3434bf1892-config\") pod \"kube-apiserver-proxy-ip-10-0-128-224.ec2.internal\" (UID: \"e9f64c09e4b0a625629b9a3434bf1892\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.693098 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.692981 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/77dcdf2501699d64463907b44b284337-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal\" (UID: \"77dcdf2501699d64463907b44b284337\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.693098 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.693006 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77dcdf2501699d64463907b44b284337-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal\" (UID: \"77dcdf2501699d64463907b44b284337\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.693098 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.693047 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77dcdf2501699d64463907b44b284337-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal\" (UID: \"77dcdf2501699d64463907b44b284337\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.693098 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.693047 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e9f64c09e4b0a625629b9a3434bf1892-config\") pod \"kube-apiserver-proxy-ip-10-0-128-224.ec2.internal\" (UID: \"e9f64c09e4b0a625629b9a3434bf1892\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.693098 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.693053 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/77dcdf2501699d64463907b44b284337-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal\" (UID: \"77dcdf2501699d64463907b44b284337\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.783500 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.783401 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:11.883957 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.883923 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:11.908561 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.908522 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.913766 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:11.913747 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal" Jul 16 00:13:11.985005 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:11.984967 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:12.085639 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:12.085565 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:12.186064 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:12.186025 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:12.232722 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.232686 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 00:13:12.233187 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.232827 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 00:13:12.286714 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:12.286681 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:12.296768 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.296731 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 00:08:11 +0000 UTC" deadline="2028-02-13 08:41:17.005900638 +0000 UTC" Jul 16 00:13:12.296768 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.296760 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13856h28m4.709145198s" Jul 16 00:13:12.328955 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.328929 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 00:13:12.387048 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:12.386982 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:12.423320 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:12.423287 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77dcdf2501699d64463907b44b284337.slice/crio-fb920c4da7b3c9fdfdceb7a4ce05042d00fccd24595d976e9d74e9cb74c13260 WatchSource:0}: Error finding container fb920c4da7b3c9fdfdceb7a4ce05042d00fccd24595d976e9d74e9cb74c13260: Status 404 returned error can't find the container with id fb920c4da7b3c9fdfdceb7a4ce05042d00fccd24595d976e9d74e9cb74c13260 Jul 16 00:13:12.426265 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:12.426239 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f64c09e4b0a625629b9a3434bf1892.slice/crio-26657215e5b88f78cf51c80d46256372941a8650ac46b2b1b38a72dae96cabd0 WatchSource:0}: Error finding container 26657215e5b88f78cf51c80d46256372941a8650ac46b2b1b38a72dae96cabd0: Status 404 returned error can't find the container with id 26657215e5b88f78cf51c80d46256372941a8650ac46b2b1b38a72dae96cabd0 Jul 16 00:13:12.430170 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.430150 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 00:13:12.482194 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.482129 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-224.ec2.internal" event={"ID":"e9f64c09e4b0a625629b9a3434bf1892","Type":"ContainerStarted","Data":"26657215e5b88f78cf51c80d46256372941a8650ac46b2b1b38a72dae96cabd0"} Jul 16 00:13:12.483157 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.483133 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal" event={"ID":"77dcdf2501699d64463907b44b284337","Type":"ContainerStarted","Data":"fb920c4da7b3c9fdfdceb7a4ce05042d00fccd24595d976e9d74e9cb74c13260"} Jul 16 00:13:12.487293 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:12.487263 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:12.574927 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.574893 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 00:13:12.587663 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:12.587637 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:12.688233 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:12.688156 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:12.789045 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:12.789009 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-224.ec2.internal\" not found" Jul 16 00:13:12.843076 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.843047 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 00:13:12.890378 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.890350 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal" Jul 16 00:13:12.901164 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.901130 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 00:13:12.902120 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.902086 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-224.ec2.internal" Jul 16 00:13:12.910397 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:12.910374 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 00:13:13.273215 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.273189 2564 apiserver.go:52] "Watching apiserver" Jul 16 00:13:13.280248 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.280221 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 00:13:13.280815 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.280787 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-r5xhf","openshift-multus/network-metrics-daemon-l725h","openshift-network-operator/iptables-alerter-w647c","openshift-ovn-kubernetes/ovnkube-node-qrwnm","kube-system/global-pull-secret-syncer-xl7gl","kube-system/konnectivity-agent-btt64","kube-system/kube-apiserver-proxy-ip-10-0-128-224.ec2.internal","openshift-dns/node-resolver-mcr99","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal","openshift-network-diagnostics/network-check-target-wzszc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8","openshift-cluster-node-tuning-operator/tuned-mk792","openshift-image-registry/node-ca-sf9f6","openshift-multus/multus-additional-cni-plugins-rltww"] Jul 16 00:13:13.282384 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.282359 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:13.282481 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.282451 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:13.283413 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.283382 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sf9f6" Jul 16 00:13:13.284834 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.284688 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w647c" Jul 16 00:13:13.285855 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.285830 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 00:13:13.285953 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.285939 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 00:13:13.286129 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.286114 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 00:13:13.286313 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.286296 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-62xmd\"" Jul 16 00:13:13.286506 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.286490 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.287309 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.287194 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 00:13:13.287475 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.287457 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 00:13:13.287596 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.287580 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7rf4t\"" Jul 16 00:13:13.288142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.287745 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 00:13:13.288142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.288014 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:13.288142 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.288087 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:13.288843 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.288649 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kwf57\"" Jul 16 00:13:13.288843 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.288728 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 00:13:13.289819 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.289613 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-btt64" Jul 16 00:13:13.289819 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.289704 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 00:13:13.289819 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.289710 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 00:13:13.289819 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.289716 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 00:13:13.290042 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.289974 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 00:13:13.290042 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.290014 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 00:13:13.290353 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.290335 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 00:13:13.291241 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.291223 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.292057 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.291906 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 00:13:13.292421 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.292376 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 00:13:13.292694 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.292674 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7bsb2\"" Jul 16 00:13:13.293120 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.293101 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 00:13:13.294893 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.294293 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 00:13:13.294893 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.294327 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vvsfj\"" Jul 16 00:13:13.294893 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.294359 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 00:13:13.294893 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.294299 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 00:13:13.295117 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.295001 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 00:13:13.296395 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.295305 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mcr99" Jul 16 00:13:13.298249 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.297911 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 00:13:13.298361 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.298322 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 00:13:13.298571 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.298419 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:13.298653 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.298514 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5vsg4\"" Jul 16 00:13:13.298775 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.298739 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:13.300975 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.300945 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.302124 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302105 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/827e2908-fba6-44a1-bfb4-6b22753d0eee-ovnkube-config\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.302203 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302139 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tpnb\" (UniqueName: \"kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb\") pod \"network-check-target-wzszc\" (UID: \"3c4e248d-a32f-48ee-a338-55864ccbfa19\") " pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:13.302203 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302162 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-cni-binary-copy\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.302309 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302230 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/868d72f4-c70a-47ab-842d-5d22e5586970-kubelet-config\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:13.302309 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302264 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-etc-openvswitch\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.302309 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302288 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-node-log\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.302446 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302325 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a49dd8c-7e28-416a-9e61-102f27d5c7df-agent-certs\") pod \"konnectivity-agent-btt64\" (UID: \"8a49dd8c-7e28-416a-9e61-102f27d5c7df\") " pod="kube-system/konnectivity-agent-btt64" Jul 16 00:13:13.302446 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302350 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.302446 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302390 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-slash\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.302446 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302429 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-run-openvswitch\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.302644 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302470 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/827e2908-fba6-44a1-bfb4-6b22753d0eee-ovnkube-script-lib\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.302644 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302493 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/63cc176e-fbac-4a75-877f-db6742663c92-host\") pod \"node-ca-sf9f6\" (UID: \"63cc176e-fbac-4a75-877f-db6742663c92\") " pod="openshift-image-registry/node-ca-sf9f6" Jul 16 00:13:13.302644 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302516 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/63cc176e-fbac-4a75-877f-db6742663c92-serviceca\") pod \"node-ca-sf9f6\" (UID: \"63cc176e-fbac-4a75-877f-db6742663c92\") " pod="openshift-image-registry/node-ca-sf9f6" Jul 16 00:13:13.302644 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302556 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-system-cni-dir\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.302644 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302573 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 00:13:13.302644 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302578 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb681aac-22c3-4d6c-b238-79d5ee473549-host-slash\") pod \"iptables-alerter-w647c\" (UID: \"eb681aac-22c3-4d6c-b238-79d5ee473549\") " pod="openshift-network-operator/iptables-alerter-w647c" Jul 16 00:13:13.302644 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302599 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-var-lib-openvswitch\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.302644 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302621 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/827e2908-fba6-44a1-bfb4-6b22753d0eee-env-overrides\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.302644 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302644 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302664 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/868d72f4-c70a-47ab-842d-5d22e5586970-dbus\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302695 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eb681aac-22c3-4d6c-b238-79d5ee473549-iptables-alerter-script\") pod \"iptables-alerter-w647c\" (UID: \"eb681aac-22c3-4d6c-b238-79d5ee473549\") " pod="openshift-network-operator/iptables-alerter-w647c" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302732 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-kubelet\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302757 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-systemd-units\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302766 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302779 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-run-netns\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302803 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-run-ovn-kubernetes\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302830 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-cni-bin\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302860 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-run-ovn\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302882 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302887 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302911 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/827e2908-fba6-44a1-bfb4-6b22753d0eee-ovn-node-metrics-cert\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302940 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzdx9\" (UniqueName: \"kubernetes.io/projected/827e2908-fba6-44a1-bfb4-6b22753d0eee-kube-api-access-fzdx9\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302962 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.302985 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-cni-netd\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.303078 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.303053 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a49dd8c-7e28-416a-9e61-102f27d5c7df-konnectivity-ca\") pod \"konnectivity-agent-btt64\" (UID: \"8a49dd8c-7e28-416a-9e61-102f27d5c7df\") " pod="kube-system/konnectivity-agent-btt64" Jul 16 00:13:13.303826 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.303095 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfvl5\" (UniqueName: \"kubernetes.io/projected/63cc176e-fbac-4a75-877f-db6742663c92-kube-api-access-zfvl5\") pod \"node-ca-sf9f6\" (UID: \"63cc176e-fbac-4a75-877f-db6742663c92\") " pod="openshift-image-registry/node-ca-sf9f6" Jul 16 00:13:13.303826 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.303102 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rlsk2\"" Jul 16 00:13:13.303826 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.303122 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-log-socket\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.303826 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.303145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-cnibin\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.303826 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.303226 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-os-release\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.303826 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.303303 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.303826 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.303328 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6drh\" (UniqueName: \"kubernetes.io/projected/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-kube-api-access-m6drh\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.303826 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.303353 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkvxl\" (UniqueName: \"kubernetes.io/projected/eb681aac-22c3-4d6c-b238-79d5ee473549-kube-api-access-vkvxl\") pod \"iptables-alerter-w647c\" (UID: \"eb681aac-22c3-4d6c-b238-79d5ee473549\") " pod="openshift-network-operator/iptables-alerter-w647c" Jul 16 00:13:13.303826 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.303375 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-run-systemd\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.304501 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.304480 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.304624 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.304608 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 00:13:13.305095 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.305049 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 00:13:13.305178 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.305123 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8b47d\"" Jul 16 00:13:13.305178 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.305169 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 00:13:13.306600 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.306581 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7ldqc\"" Jul 16 00:13:13.306804 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.306771 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 00:13:13.306804 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.306785 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 00:13:13.326987 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.326961 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cj5hh" Jul 16 00:13:13.332137 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.332028 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cj5hh" Jul 16 00:13:13.391159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.391132 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 00:13:13.404092 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404052 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-run-openvswitch\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.404269 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404102 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/63cc176e-fbac-4a75-877f-db6742663c92-host\") pod \"node-ca-sf9f6\" (UID: \"63cc176e-fbac-4a75-877f-db6742663c92\") " pod="openshift-image-registry/node-ca-sf9f6" Jul 16 00:13:13.404269 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404139 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/63cc176e-fbac-4a75-877f-db6742663c92-serviceca\") pod \"node-ca-sf9f6\" (UID: \"63cc176e-fbac-4a75-877f-db6742663c92\") " pod="openshift-image-registry/node-ca-sf9f6" Jul 16 00:13:13.404269 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404159 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-system-cni-dir\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.404269 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404169 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/63cc176e-fbac-4a75-877f-db6742663c92-host\") pod \"node-ca-sf9f6\" (UID: \"63cc176e-fbac-4a75-877f-db6742663c92\") " pod="openshift-image-registry/node-ca-sf9f6" Jul 16 00:13:13.404269 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404184 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-sysctl-conf\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.404269 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404222 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/827e2908-fba6-44a1-bfb4-6b22753d0eee-env-overrides\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.404269 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404253 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-run-openvswitch\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.404269 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404253 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.404676 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404290 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-tuned\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.404676 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404312 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-etc-kubernetes\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.404676 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404337 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plgzg\" (UniqueName: \"kubernetes.io/projected/d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c-kube-api-access-plgzg\") pod \"node-resolver-mcr99\" (UID: \"d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c\") " pod="openshift-dns/node-resolver-mcr99" Jul 16 00:13:13.404676 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404385 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eb681aac-22c3-4d6c-b238-79d5ee473549-iptables-alerter-script\") pod \"iptables-alerter-w647c\" (UID: \"eb681aac-22c3-4d6c-b238-79d5ee473549\") " pod="openshift-network-operator/iptables-alerter-w647c" Jul 16 00:13:13.404676 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404410 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-kubelet\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.404676 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404442 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-systemd-units\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.404676 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404476 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-run-ovn-kubernetes\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.404676 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404500 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.405068 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404778 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/827e2908-fba6-44a1-bfb4-6b22753d0eee-env-overrides\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.405068 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404779 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/63cc176e-fbac-4a75-877f-db6742663c92-serviceca\") pod \"node-ca-sf9f6\" (UID: \"63cc176e-fbac-4a75-877f-db6742663c92\") " pod="openshift-image-registry/node-ca-sf9f6" Jul 16 00:13:13.405068 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404816 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-system-cni-dir\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.405068 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404851 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.405068 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404861 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-systemd-units\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.405068 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404884 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-kubelet\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.405068 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404896 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-run-ovn-kubernetes\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.405068 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404968 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.405068 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.404994 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzdx9\" (UniqueName: \"kubernetes.io/projected/827e2908-fba6-44a1-bfb4-6b22753d0eee-kube-api-access-fzdx9\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.405068 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405036 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eb681aac-22c3-4d6c-b238-79d5ee473549-iptables-alerter-script\") pod \"iptables-alerter-w647c\" (UID: \"eb681aac-22c3-4d6c-b238-79d5ee473549\") " pod="openshift-network-operator/iptables-alerter-w647c" Jul 16 00:13:13.405068 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405052 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6487ac13-1c94-4d7f-917b-f2f14892e58d-multus-daemon-config\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.405487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405077 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/827e2908-fba6-44a1-bfb4-6b22753d0eee-ovn-node-metrics-cert\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.405487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405098 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-multus-cni-dir\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.405487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405179 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4vz6\" (UniqueName: \"kubernetes.io/projected/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-kube-api-access-w4vz6\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:13.405487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405207 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-registration-dir\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.405487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405256 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-cni-netd\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.405487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405294 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfvl5\" (UniqueName: \"kubernetes.io/projected/63cc176e-fbac-4a75-877f-db6742663c92-kube-api-access-zfvl5\") pod \"node-ca-sf9f6\" (UID: \"63cc176e-fbac-4a75-877f-db6742663c92\") " pod="openshift-image-registry/node-ca-sf9f6" Jul 16 00:13:13.405487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405328 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-cni-netd\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.405487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405364 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-sysctl-d\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.405487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405384 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-run\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.405487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405412 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-var-lib-kubelet\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.405487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405445 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-host\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.405487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405490 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-cnibin\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.405487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405495 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405520 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-os-release\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405562 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405586 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-modprobe-d\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405594 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-cnibin\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405718 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-os-release\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405758 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-multus-conf-dir\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405799 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkvxl\" (UniqueName: \"kubernetes.io/projected/eb681aac-22c3-4d6c-b238-79d5ee473549-kube-api-access-vkvxl\") pod \"iptables-alerter-w647c\" (UID: \"eb681aac-22c3-4d6c-b238-79d5ee473549\") " pod="openshift-network-operator/iptables-alerter-w647c" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405834 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tpnb\" (UniqueName: \"kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb\") pod \"network-check-target-wzszc\" (UID: \"3c4e248d-a32f-48ee-a338-55864ccbfa19\") " pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405856 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-sysconfig\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405928 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/375a83e3-da87-4562-b809-09a2f4fa9f22-tmp\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405962 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-system-cni-dir\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405985 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406010 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-node-log\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406063 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.406112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406086 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.405645 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-os-release\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406098 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-run-k8s-cni-cncf-io\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406166 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-node-log\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406208 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-sys-fs\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406230 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406257 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-slash\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406298 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/827e2908-fba6-44a1-bfb4-6b22753d0eee-ovnkube-script-lib\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406324 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-slash\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406324 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-sys\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406358 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-hostroot\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406397 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-device-dir\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406430 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb681aac-22c3-4d6c-b238-79d5ee473549-host-slash\") pod \"iptables-alerter-w647c\" (UID: \"eb681aac-22c3-4d6c-b238-79d5ee473549\") " pod="openshift-network-operator/iptables-alerter-w647c" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406452 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-var-lib-openvswitch\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406510 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb681aac-22c3-4d6c-b238-79d5ee473549-host-slash\") pod \"iptables-alerter-w647c\" (UID: \"eb681aac-22c3-4d6c-b238-79d5ee473549\") " pod="openshift-network-operator/iptables-alerter-w647c" Jul 16 00:13:13.406835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.406559 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-cnibin\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.407467 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407001 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-var-lib-openvswitch\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.407467 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407066 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-multus-socket-dir-parent\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.407467 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407107 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-var-lib-cni-bin\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.407467 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407172 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-var-lib-cni-multus\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.407467 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407220 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/868d72f4-c70a-47ab-842d-5d22e5586970-dbus\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:13.407467 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407264 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-run-netns\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.407467 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407299 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-cni-bin\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.407467 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407338 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:13.407467 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407381 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-systemd\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.407467 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407411 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4qnk\" (UniqueName: \"kubernetes.io/projected/375a83e3-da87-4562-b809-09a2f4fa9f22-kube-api-access-g4qnk\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.407933 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407518 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/868d72f4-c70a-47ab-842d-5d22e5586970-dbus\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:13.407933 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407560 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/827e2908-fba6-44a1-bfb4-6b22753d0eee-ovnkube-script-lib\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.407933 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407566 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-run-netns\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.407933 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.407656 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:13.407933 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407473 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-socket-dir\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.407933 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407710 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-host-cni-bin\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.407933 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.407816 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret podName:868d72f4-c70a-47ab-842d-5d22e5586970 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:13.907795418 +0000 UTC m=+2.975735893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret") pod "global-pull-secret-syncer-xl7gl" (UID: "868d72f4-c70a-47ab-842d-5d22e5586970") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:13.407933 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407837 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-run-ovn\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.407933 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407870 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-lib-modules\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.407933 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407902 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-var-lib-kubelet\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.407933 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407905 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-run-ovn\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.407933 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407933 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c-hosts-file\") pod \"node-resolver-mcr99\" (UID: \"d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c\") " pod="openshift-dns/node-resolver-mcr99" Jul 16 00:13:13.408543 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.407974 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a49dd8c-7e28-416a-9e61-102f27d5c7df-konnectivity-ca\") pod \"konnectivity-agent-btt64\" (UID: \"8a49dd8c-7e28-416a-9e61-102f27d5c7df\") " pod="kube-system/konnectivity-agent-btt64" Jul 16 00:13:13.408543 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408035 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-kubernetes\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.408543 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408068 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.408543 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408113 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7nxj\" (UniqueName: \"kubernetes.io/projected/4a0f53d2-f79b-4267-843a-4b58096fc425-kube-api-access-n7nxj\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.408543 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408184 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-log-socket\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.408543 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408246 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-log-socket\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.408543 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408255 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6drh\" (UniqueName: \"kubernetes.io/projected/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-kube-api-access-m6drh\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.408543 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408318 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-run-netns\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.408543 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408353 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c-tmp-dir\") pod \"node-resolver-mcr99\" (UID: \"d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c\") " pod="openshift-dns/node-resolver-mcr99" Jul 16 00:13:13.408543 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408453 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-run-systemd\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.408543 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408493 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/827e2908-fba6-44a1-bfb4-6b22753d0eee-ovnkube-config\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.409024 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408551 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-cni-binary-copy\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.409024 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408557 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a49dd8c-7e28-416a-9e61-102f27d5c7df-konnectivity-ca\") pod \"konnectivity-agent-btt64\" (UID: \"8a49dd8c-7e28-416a-9e61-102f27d5c7df\") " pod="kube-system/konnectivity-agent-btt64" Jul 16 00:13:13.409024 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408577 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/868d72f4-c70a-47ab-842d-5d22e5586970-kubelet-config\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:13.409024 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408845 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-etc-openvswitch\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.409024 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408877 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a49dd8c-7e28-416a-9e61-102f27d5c7df-agent-certs\") pod \"konnectivity-agent-btt64\" (UID: \"8a49dd8c-7e28-416a-9e61-102f27d5c7df\") " pod="kube-system/konnectivity-agent-btt64" Jul 16 00:13:13.409024 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408907 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6487ac13-1c94-4d7f-917b-f2f14892e58d-cni-binary-copy\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.409024 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408942 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-run-multus-certs\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.409024 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.408972 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzp5b\" (UniqueName: \"kubernetes.io/projected/6487ac13-1c94-4d7f-917b-f2f14892e58d-kube-api-access-kzp5b\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.409024 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.409003 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-etc-selinux\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.409398 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.409352 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/827e2908-fba6-44a1-bfb4-6b22753d0eee-ovnkube-config\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.409398 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.409355 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/868d72f4-c70a-47ab-842d-5d22e5586970-kubelet-config\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:13.409398 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.409392 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-etc-openvswitch\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.409570 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.409450 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/827e2908-fba6-44a1-bfb4-6b22753d0eee-run-systemd\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.409878 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.409826 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-cni-binary-copy\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.409996 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.409976 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/827e2908-fba6-44a1-bfb4-6b22753d0eee-ovn-node-metrics-cert\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.411518 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.411487 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:13:13.411677 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.411663 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:13:13.411764 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.411755 2564 projected.go:196] Error preparing data for projected volume kube-api-access-9tpnb for pod openshift-network-diagnostics/network-check-target-wzszc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:13.411898 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.411885 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb podName:3c4e248d-a32f-48ee-a338-55864ccbfa19 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:13.911868227 +0000 UTC m=+2.979808680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9tpnb" (UniqueName: "kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb") pod "network-check-target-wzszc" (UID: "3c4e248d-a32f-48ee-a338-55864ccbfa19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:13.412212 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.412190 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a49dd8c-7e28-416a-9e61-102f27d5c7df-agent-certs\") pod \"konnectivity-agent-btt64\" (UID: \"8a49dd8c-7e28-416a-9e61-102f27d5c7df\") " pod="kube-system/konnectivity-agent-btt64" Jul 16 00:13:13.414211 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.414064 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzdx9\" (UniqueName: \"kubernetes.io/projected/827e2908-fba6-44a1-bfb4-6b22753d0eee-kube-api-access-fzdx9\") pod \"ovnkube-node-qrwnm\" (UID: \"827e2908-fba6-44a1-bfb4-6b22753d0eee\") " pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.414426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.414406 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkvxl\" (UniqueName: \"kubernetes.io/projected/eb681aac-22c3-4d6c-b238-79d5ee473549-kube-api-access-vkvxl\") pod \"iptables-alerter-w647c\" (UID: \"eb681aac-22c3-4d6c-b238-79d5ee473549\") " pod="openshift-network-operator/iptables-alerter-w647c" Jul 16 00:13:13.415037 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.415017 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfvl5\" (UniqueName: \"kubernetes.io/projected/63cc176e-fbac-4a75-877f-db6742663c92-kube-api-access-zfvl5\") pod \"node-ca-sf9f6\" (UID: \"63cc176e-fbac-4a75-877f-db6742663c92\") " pod="openshift-image-registry/node-ca-sf9f6" Jul 16 00:13:13.418216 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.418193 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6drh\" (UniqueName: \"kubernetes.io/projected/4089f83d-6157-48c3-8ed0-aa5bf7f01f7b-kube-api-access-m6drh\") pod \"multus-additional-cni-plugins-rltww\" (UID: \"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b\") " pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.509898 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.509859 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c-hosts-file\") pod \"node-resolver-mcr99\" (UID: \"d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c\") " pod="openshift-dns/node-resolver-mcr99" Jul 16 00:13:13.509898 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.509903 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-kubernetes\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.510128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.509924 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.510128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.509945 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7nxj\" (UniqueName: \"kubernetes.io/projected/4a0f53d2-f79b-4267-843a-4b58096fc425-kube-api-access-n7nxj\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.510128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.509968 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-run-netns\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.510128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.509988 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c-tmp-dir\") pod \"node-resolver-mcr99\" (UID: \"d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c\") " pod="openshift-dns/node-resolver-mcr99" Jul 16 00:13:13.510128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510006 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-kubernetes\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.510128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510014 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.510128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510002 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c-hosts-file\") pod \"node-resolver-mcr99\" (UID: \"d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c\") " pod="openshift-dns/node-resolver-mcr99" Jul 16 00:13:13.510128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510013 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6487ac13-1c94-4d7f-917b-f2f14892e58d-cni-binary-copy\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.510128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510077 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-run-multus-certs\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.510128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510103 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzp5b\" (UniqueName: \"kubernetes.io/projected/6487ac13-1c94-4d7f-917b-f2f14892e58d-kube-api-access-kzp5b\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.510128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510038 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-run-netns\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.510128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510124 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-etc-selinux\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510152 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-sysctl-conf\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510175 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-tuned\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510180 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-run-multus-certs\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510196 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-etc-kubernetes\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510224 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-etc-kubernetes\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510244 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plgzg\" (UniqueName: \"kubernetes.io/projected/d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c-kube-api-access-plgzg\") pod \"node-resolver-mcr99\" (UID: \"d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c\") " pod="openshift-dns/node-resolver-mcr99" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510277 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6487ac13-1c94-4d7f-917b-f2f14892e58d-multus-daemon-config\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510264 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-etc-selinux\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510304 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-multus-cni-dir\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510326 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vz6\" (UniqueName: \"kubernetes.io/projected/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-kube-api-access-w4vz6\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510351 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-registration-dir\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510355 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-sysctl-conf\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510364 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c-tmp-dir\") pod \"node-resolver-mcr99\" (UID: \"d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c\") " pod="openshift-dns/node-resolver-mcr99" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510386 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-sysctl-d\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510424 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-run\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510434 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-multus-cni-dir\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510449 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-var-lib-kubelet\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.510691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510472 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-host\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510489 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-sysctl-d\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510500 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-modprobe-d\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510555 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-os-release\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510578 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-multus-conf-dir\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510586 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-registration-dir\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510599 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-var-lib-kubelet\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510617 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-run\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510627 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-multus-conf-dir\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510626 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-sysconfig\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510665 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-host\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510674 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-sysconfig\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510678 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-modprobe-d\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510681 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6487ac13-1c94-4d7f-917b-f2f14892e58d-cni-binary-copy\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510743 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-os-release\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510725 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/375a83e3-da87-4562-b809-09a2f4fa9f22-tmp\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510781 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-system-cni-dir\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510810 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510853 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-system-cni-dir\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.511426 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.510898 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510951 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-run-k8s-cni-cncf-io\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.510964 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs podName:5f9ca78d-1eb6-45bc-bf9a-01b646191c67 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:14.010945063 +0000 UTC m=+3.078885526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs") pod "network-metrics-daemon-l725h" (UID: "5f9ca78d-1eb6-45bc-bf9a-01b646191c67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510971 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6487ac13-1c94-4d7f-917b-f2f14892e58d-multus-daemon-config\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-sys-fs\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511019 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-sys\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.510988 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-run-k8s-cni-cncf-io\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511041 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-hostroot\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511072 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-hostroot\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511078 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-device-dir\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511103 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-sys\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511111 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-cnibin\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511129 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-device-dir\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511146 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-multus-socket-dir-parent\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511165 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-sys-fs\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511167 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-var-lib-cni-bin\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511167 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-cnibin\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511205 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-var-lib-cni-multus\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512875 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511205 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-var-lib-cni-bin\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512875 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511219 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-multus-socket-dir-parent\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512875 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511248 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-var-lib-cni-multus\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512875 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511259 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-systemd\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.512875 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511287 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4qnk\" (UniqueName: \"kubernetes.io/projected/375a83e3-da87-4562-b809-09a2f4fa9f22-kube-api-access-g4qnk\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.512875 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511328 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-socket-dir\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.512875 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511356 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-systemd\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.512875 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511352 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-lib-modules\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.512875 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511388 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-var-lib-kubelet\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512875 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511463 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/375a83e3-da87-4562-b809-09a2f4fa9f22-lib-modules\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.512875 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511478 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6487ac13-1c94-4d7f-917b-f2f14892e58d-host-var-lib-kubelet\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.512875 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.511548 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4a0f53d2-f79b-4267-843a-4b58096fc425-socket-dir\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.513298 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.512924 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/375a83e3-da87-4562-b809-09a2f4fa9f22-etc-tuned\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.513298 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.513130 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/375a83e3-da87-4562-b809-09a2f4fa9f22-tmp\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.517636 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.517608 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7nxj\" (UniqueName: \"kubernetes.io/projected/4a0f53d2-f79b-4267-843a-4b58096fc425-kube-api-access-n7nxj\") pod \"aws-ebs-csi-driver-node-xcsf8\" (UID: \"4a0f53d2-f79b-4267-843a-4b58096fc425\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.519363 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.519339 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plgzg\" (UniqueName: \"kubernetes.io/projected/d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c-kube-api-access-plgzg\") pod \"node-resolver-mcr99\" (UID: \"d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c\") " pod="openshift-dns/node-resolver-mcr99" Jul 16 00:13:13.521446 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.521425 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4vz6\" (UniqueName: \"kubernetes.io/projected/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-kube-api-access-w4vz6\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:13.521607 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.521590 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzp5b\" (UniqueName: \"kubernetes.io/projected/6487ac13-1c94-4d7f-917b-f2f14892e58d-kube-api-access-kzp5b\") pod \"multus-r5xhf\" (UID: \"6487ac13-1c94-4d7f-917b-f2f14892e58d\") " pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.521769 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.521750 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4qnk\" (UniqueName: \"kubernetes.io/projected/375a83e3-da87-4562-b809-09a2f4fa9f22-kube-api-access-g4qnk\") pod \"tuned-mk792\" (UID: \"375a83e3-da87-4562-b809-09a2f4fa9f22\") " pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.596757 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.596679 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sf9f6" Jul 16 00:13:13.606326 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.606305 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w647c" Jul 16 00:13:13.613984 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.613959 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:13.618546 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.618517 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-btt64" Jul 16 00:13:13.628913 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.628888 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rltww" Jul 16 00:13:13.635811 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.635781 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mcr99" Jul 16 00:13:13.642434 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.642407 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r5xhf" Jul 16 00:13:13.647283 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.647266 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" Jul 16 00:13:13.652882 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.652863 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mk792" Jul 16 00:13:13.785916 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.785884 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 00:13:13.915146 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.915078 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tpnb\" (UniqueName: \"kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb\") pod \"network-check-target-wzszc\" (UID: \"3c4e248d-a32f-48ee-a338-55864ccbfa19\") " pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:13.915146 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:13.915135 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:13.915304 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.915233 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:13.915304 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.915235 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:13:13.915304 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.915254 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:13:13.915304 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.915263 2564 projected.go:196] Error preparing data for projected volume kube-api-access-9tpnb for pod openshift-network-diagnostics/network-check-target-wzszc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:13.915304 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.915283 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret podName:868d72f4-c70a-47ab-842d-5d22e5586970 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:14.915268502 +0000 UTC m=+3.983208959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret") pod "global-pull-secret-syncer-xl7gl" (UID: "868d72f4-c70a-47ab-842d-5d22e5586970") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:13.915453 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:13.915305 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb podName:3c4e248d-a32f-48ee-a338-55864ccbfa19 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:14.915290262 +0000 UTC m=+3.983230728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9tpnb" (UniqueName: "kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb") pod "network-check-target-wzszc" (UID: "3c4e248d-a32f-48ee-a338-55864ccbfa19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:14.016303 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.016270 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:14.016472 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:14.016392 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:14.016472 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:14.016454 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs podName:5f9ca78d-1eb6-45bc-bf9a-01b646191c67 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:15.016440748 +0000 UTC m=+4.084381201 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs") pod "network-metrics-daemon-l725h" (UID: "5f9ca78d-1eb6-45bc-bf9a-01b646191c67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:14.272404 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:14.272380 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod375a83e3_da87_4562_b809_09a2f4fa9f22.slice/crio-3ed977ee9e39c8f13752f32f8997b23cd775a692c0e348afb4389b6d5249986c WatchSource:0}: Error finding container 3ed977ee9e39c8f13752f32f8997b23cd775a692c0e348afb4389b6d5249986c: Status 404 returned error can't find the container with id 3ed977ee9e39c8f13752f32f8997b23cd775a692c0e348afb4389b6d5249986c Jul 16 00:13:14.273035 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:14.273010 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6487ac13_1c94_4d7f_917b_f2f14892e58d.slice/crio-42715f2a560b9bcd456784891defa4166fb4df163b92d888d022f2b6bd477958 WatchSource:0}: Error finding container 42715f2a560b9bcd456784891defa4166fb4df163b92d888d022f2b6bd477958: Status 404 returned error can't find the container with id 42715f2a560b9bcd456784891defa4166fb4df163b92d888d022f2b6bd477958 Jul 16 00:13:14.274182 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:14.274087 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5ea17c6_9c8f_4862_81f4_3b6c3b380f4c.slice/crio-6c7a258eec8cbda700f7644279f9ed85c7e13c1ff14f8a6fbc89221932d55467 WatchSource:0}: Error finding container 6c7a258eec8cbda700f7644279f9ed85c7e13c1ff14f8a6fbc89221932d55467: Status 404 returned error can't find the container with id 6c7a258eec8cbda700f7644279f9ed85c7e13c1ff14f8a6fbc89221932d55467 Jul 16 00:13:14.275451 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:14.275389 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63cc176e_fbac_4a75_877f_db6742663c92.slice/crio-f00a2aa1e21f3028f24ce6d2e80aa4e2cced974a9513c604c77fbfc885de9e8c WatchSource:0}: Error finding container f00a2aa1e21f3028f24ce6d2e80aa4e2cced974a9513c604c77fbfc885de9e8c: Status 404 returned error can't find the container with id f00a2aa1e21f3028f24ce6d2e80aa4e2cced974a9513c604c77fbfc885de9e8c Jul 16 00:13:14.276034 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:14.276014 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a49dd8c_7e28_416a_9e61_102f27d5c7df.slice/crio-74855c6d9b251905e33ff5efba35c7779afddb3e717f37712b1fd7857c896925 WatchSource:0}: Error finding container 74855c6d9b251905e33ff5efba35c7779afddb3e717f37712b1fd7857c896925: Status 404 returned error can't find the container with id 74855c6d9b251905e33ff5efba35c7779afddb3e717f37712b1fd7857c896925 Jul 16 00:13:14.278527 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:14.278357 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod827e2908_fba6_44a1_bfb4_6b22753d0eee.slice/crio-0b09dc4e6c107d6fec3802de142b33d171d13c2f1afed4348a1db25470387359 WatchSource:0}: Error finding container 0b09dc4e6c107d6fec3802de142b33d171d13c2f1afed4348a1db25470387359: Status 404 returned error can't find the container with id 0b09dc4e6c107d6fec3802de142b33d171d13c2f1afed4348a1db25470387359 Jul 16 00:13:14.280468 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:14.280450 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb681aac_22c3_4d6c_b238_79d5ee473549.slice/crio-2980e5057788c988e80d3ea474ef0bfbfcfd83540ebd60d925ef0ffca31fbfcf WatchSource:0}: Error finding container 2980e5057788c988e80d3ea474ef0bfbfcfd83540ebd60d925ef0ffca31fbfcf: Status 404 returned error can't find the container with id 2980e5057788c988e80d3ea474ef0bfbfcfd83540ebd60d925ef0ffca31fbfcf Jul 16 00:13:14.281660 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:14.281638 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a0f53d2_f79b_4267_843a_4b58096fc425.slice/crio-e75693a920c8bb0b304deab62227ce725b01137e03c42d37bb56a2723f5cbdd2 WatchSource:0}: Error finding container e75693a920c8bb0b304deab62227ce725b01137e03c42d37bb56a2723f5cbdd2: Status 404 returned error can't find the container with id e75693a920c8bb0b304deab62227ce725b01137e03c42d37bb56a2723f5cbdd2 Jul 16 00:13:14.313651 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:14.313602 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4089f83d_6157_48c3_8ed0_aa5bf7f01f7b.slice/crio-2b5a01d199987944fa1d757c15eee254bd800cb6d7ab67567b567e3b1a000364 WatchSource:0}: Error finding container 2b5a01d199987944fa1d757c15eee254bd800cb6d7ab67567b567e3b1a000364: Status 404 returned error can't find the container with id 2b5a01d199987944fa1d757c15eee254bd800cb6d7ab67567b567e3b1a000364 Jul 16 00:13:14.335007 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.334973 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 00:08:13 +0000 UTC" deadline="2028-04-21 22:03:55.226138976 +0000 UTC" Jul 16 00:13:14.335007 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.335004 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15501h50m40.891138294s" Jul 16 00:13:14.479281 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.479101 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:14.479416 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.479113 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:14.479416 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:14.479393 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:14.479507 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:14.479434 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:14.486886 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.486858 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mcr99" event={"ID":"d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c","Type":"ContainerStarted","Data":"6c7a258eec8cbda700f7644279f9ed85c7e13c1ff14f8a6fbc89221932d55467"} Jul 16 00:13:14.487967 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.487943 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-btt64" event={"ID":"8a49dd8c-7e28-416a-9e61-102f27d5c7df","Type":"ContainerStarted","Data":"74855c6d9b251905e33ff5efba35c7779afddb3e717f37712b1fd7857c896925"} Jul 16 00:13:14.489748 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.489722 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-224.ec2.internal" event={"ID":"e9f64c09e4b0a625629b9a3434bf1892","Type":"ContainerStarted","Data":"cf0ed6c96342112f309645dc30640ddec068c2419fd2fd3b406f2d08d1630a27"} Jul 16 00:13:14.490851 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.490830 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w647c" event={"ID":"eb681aac-22c3-4d6c-b238-79d5ee473549","Type":"ContainerStarted","Data":"2980e5057788c988e80d3ea474ef0bfbfcfd83540ebd60d925ef0ffca31fbfcf"} Jul 16 00:13:14.495041 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.495006 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mk792" event={"ID":"375a83e3-da87-4562-b809-09a2f4fa9f22","Type":"ContainerStarted","Data":"3ed977ee9e39c8f13752f32f8997b23cd775a692c0e348afb4389b6d5249986c"} Jul 16 00:13:14.496276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.496256 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rltww" event={"ID":"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b","Type":"ContainerStarted","Data":"2b5a01d199987944fa1d757c15eee254bd800cb6d7ab67567b567e3b1a000364"} Jul 16 00:13:14.497343 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.497324 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" event={"ID":"4a0f53d2-f79b-4267-843a-4b58096fc425","Type":"ContainerStarted","Data":"e75693a920c8bb0b304deab62227ce725b01137e03c42d37bb56a2723f5cbdd2"} Jul 16 00:13:14.498280 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.498259 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r5xhf" event={"ID":"6487ac13-1c94-4d7f-917b-f2f14892e58d","Type":"ContainerStarted","Data":"42715f2a560b9bcd456784891defa4166fb4df163b92d888d022f2b6bd477958"} Jul 16 00:13:14.499305 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.499286 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" event={"ID":"827e2908-fba6-44a1-bfb4-6b22753d0eee","Type":"ContainerStarted","Data":"0b09dc4e6c107d6fec3802de142b33d171d13c2f1afed4348a1db25470387359"} Jul 16 00:13:14.500279 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.500262 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sf9f6" event={"ID":"63cc176e-fbac-4a75-877f-db6742663c92","Type":"ContainerStarted","Data":"f00a2aa1e21f3028f24ce6d2e80aa4e2cced974a9513c604c77fbfc885de9e8c"} Jul 16 00:13:14.505399 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.505325 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-224.ec2.internal" podStartSLOduration=2.50531229 podStartE2EDuration="2.50531229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:13:14.504822533 +0000 UTC m=+3.572763009" watchObservedRunningTime="2026-07-16 00:13:14.50531229 +0000 UTC m=+3.573252764" Jul 16 00:13:14.924166 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.924125 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tpnb\" (UniqueName: \"kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb\") pod \"network-check-target-wzszc\" (UID: \"3c4e248d-a32f-48ee-a338-55864ccbfa19\") " pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:14.924360 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:14.924206 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:14.924422 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:14.924404 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:14.924472 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:14.924465 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret podName:868d72f4-c70a-47ab-842d-5d22e5586970 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:16.924447984 +0000 UTC m=+5.992388442 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret") pod "global-pull-secret-syncer-xl7gl" (UID: "868d72f4-c70a-47ab-842d-5d22e5586970") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:14.924834 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:14.924685 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:13:14.924834 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:14.924712 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:13:14.924834 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:14.924725 2564 projected.go:196] Error preparing data for projected volume kube-api-access-9tpnb for pod openshift-network-diagnostics/network-check-target-wzszc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:14.924834 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:14.924790 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb podName:3c4e248d-a32f-48ee-a338-55864ccbfa19 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:16.924760651 +0000 UTC m=+5.992701110 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9tpnb" (UniqueName: "kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb") pod "network-check-target-wzszc" (UID: "3c4e248d-a32f-48ee-a338-55864ccbfa19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:15.025396 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:15.024786 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:15.025396 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:15.024956 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:15.025396 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:15.025022 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs podName:5f9ca78d-1eb6-45bc-bf9a-01b646191c67 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:17.025001661 +0000 UTC m=+6.092942117 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs") pod "network-metrics-daemon-l725h" (UID: "5f9ca78d-1eb6-45bc-bf9a-01b646191c67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:15.336129 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:15.336059 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 00:08:13 +0000 UTC" deadline="2028-04-14 09:07:24.122460102 +0000 UTC" Jul 16 00:13:15.336129 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:15.336120 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15320h54m8.786344232s" Jul 16 00:13:15.481364 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:15.481287 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:15.481522 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:15.481420 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:15.515521 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:15.515469 2564 generic.go:356] "Generic (PLEG): container finished" podID="77dcdf2501699d64463907b44b284337" containerID="b331e4bbc7d23b752238417cf34631d3542a8909c3dfedbdcef7f6b1762b5098" exitCode=0 Jul 16 00:13:15.515736 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:15.515700 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal" event={"ID":"77dcdf2501699d64463907b44b284337","Type":"ContainerDied","Data":"b331e4bbc7d23b752238417cf34631d3542a8909c3dfedbdcef7f6b1762b5098"} Jul 16 00:13:16.479133 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:16.479097 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:16.479615 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:16.479244 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:16.479721 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:16.479700 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:16.479839 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:16.479809 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:16.539249 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:16.538506 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal" event={"ID":"77dcdf2501699d64463907b44b284337","Type":"ContainerStarted","Data":"3703a38760e0ae05d7bcb1dca2b20b7671ba040838a375324325da2726bb12bc"} Jul 16 00:13:16.940893 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:16.940436 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tpnb\" (UniqueName: \"kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb\") pod \"network-check-target-wzszc\" (UID: \"3c4e248d-a32f-48ee-a338-55864ccbfa19\") " pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:16.940893 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:16.940504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:16.940893 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:16.940689 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:16.940893 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:16.940797 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:13:16.940893 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:16.940826 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:13:16.940893 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:16.940840 2564 projected.go:196] Error preparing data for projected volume kube-api-access-9tpnb for pod openshift-network-diagnostics/network-check-target-wzszc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:16.940893 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:16.940801 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret podName:868d72f4-c70a-47ab-842d-5d22e5586970 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:20.94074446 +0000 UTC m=+10.008684913 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret") pod "global-pull-secret-syncer-xl7gl" (UID: "868d72f4-c70a-47ab-842d-5d22e5586970") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:16.940893 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:16.940903 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb podName:3c4e248d-a32f-48ee-a338-55864ccbfa19 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:20.940886902 +0000 UTC m=+10.008827356 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9tpnb" (UniqueName: "kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb") pod "network-check-target-wzszc" (UID: "3c4e248d-a32f-48ee-a338-55864ccbfa19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:17.041951 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:17.041338 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:17.041951 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:17.041546 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:17.041951 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:17.041611 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs podName:5f9ca78d-1eb6-45bc-bf9a-01b646191c67 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:21.04159207 +0000 UTC m=+10.109532526 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs") pod "network-metrics-daemon-l725h" (UID: "5f9ca78d-1eb6-45bc-bf9a-01b646191c67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:17.481416 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:17.481379 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:17.482064 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:17.481508 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:18.479274 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:18.479237 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:18.479505 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:18.479237 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:18.479505 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:18.479380 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:18.479505 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:18.479415 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:19.479348 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:19.479308 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:19.479809 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:19.479455 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:20.479342 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:20.479282 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:20.479559 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:20.479433 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:20.479953 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:20.479287 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:20.479953 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:20.479898 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:20.975743 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:20.975680 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tpnb\" (UniqueName: \"kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb\") pod \"network-check-target-wzszc\" (UID: \"3c4e248d-a32f-48ee-a338-55864ccbfa19\") " pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:20.975743 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:20.975750 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:20.976015 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:20.975897 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:20.976015 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:20.975961 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret podName:868d72f4-c70a-47ab-842d-5d22e5586970 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:28.975942636 +0000 UTC m=+18.043883091 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret") pod "global-pull-secret-syncer-xl7gl" (UID: "868d72f4-c70a-47ab-842d-5d22e5586970") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:20.976390 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:20.976270 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:13:20.976390 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:20.976296 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:13:20.976390 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:20.976316 2564 projected.go:196] Error preparing data for projected volume kube-api-access-9tpnb for pod openshift-network-diagnostics/network-check-target-wzszc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:20.976390 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:20.976371 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb podName:3c4e248d-a32f-48ee-a338-55864ccbfa19 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:28.976355706 +0000 UTC m=+18.044296161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9tpnb" (UniqueName: "kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb") pod "network-check-target-wzszc" (UID: "3c4e248d-a32f-48ee-a338-55864ccbfa19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:21.077119 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:21.077061 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:21.077289 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:21.077248 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:21.077347 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:21.077331 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs podName:5f9ca78d-1eb6-45bc-bf9a-01b646191c67 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:29.077308849 +0000 UTC m=+18.145249317 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs") pod "network-metrics-daemon-l725h" (UID: "5f9ca78d-1eb6-45bc-bf9a-01b646191c67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:21.480799 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:21.480284 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:21.480799 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:21.480396 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:22.478552 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:22.478499 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:22.478552 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:22.478527 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:22.478780 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:22.478636 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:22.479000 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:22.478958 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:23.479359 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:23.479317 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:23.479870 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:23.479461 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:24.479319 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:24.479286 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:24.479475 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:24.479325 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:24.479475 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:24.479405 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:24.479860 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:24.479517 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:25.479565 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:25.479516 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:25.480001 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:25.479646 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:26.478872 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:26.478836 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:26.478872 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:26.478887 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:26.479092 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:26.478978 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:26.479167 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:26.479133 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:27.478961 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:27.478925 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:27.479440 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:27.479051 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:28.478622 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:28.478580 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:28.478885 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:28.478629 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:28.478885 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:28.478724 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:28.478885 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:28.478861 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:29.039767 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:29.039730 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tpnb\" (UniqueName: \"kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb\") pod \"network-check-target-wzszc\" (UID: \"3c4e248d-a32f-48ee-a338-55864ccbfa19\") " pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:29.040269 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:29.039796 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:29.040269 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:29.039927 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:29.040269 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:29.039936 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:13:29.040269 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:29.039964 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:13:29.040269 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:29.039975 2564 projected.go:196] Error preparing data for projected volume kube-api-access-9tpnb for pod openshift-network-diagnostics/network-check-target-wzszc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:29.040269 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:29.039992 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret podName:868d72f4-c70a-47ab-842d-5d22e5586970 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:45.039975686 +0000 UTC m=+34.107916141 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret") pod "global-pull-secret-syncer-xl7gl" (UID: "868d72f4-c70a-47ab-842d-5d22e5586970") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:29.040269 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:29.040015 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb podName:3c4e248d-a32f-48ee-a338-55864ccbfa19 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:45.040002459 +0000 UTC m=+34.107942941 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9tpnb" (UniqueName: "kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb") pod "network-check-target-wzszc" (UID: "3c4e248d-a32f-48ee-a338-55864ccbfa19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:29.140357 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:29.140316 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:29.140526 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:29.140498 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:29.140625 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:29.140591 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs podName:5f9ca78d-1eb6-45bc-bf9a-01b646191c67 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:45.140574365 +0000 UTC m=+34.208514837 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs") pod "network-metrics-daemon-l725h" (UID: "5f9ca78d-1eb6-45bc-bf9a-01b646191c67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:29.478646 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:29.478545 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:29.478800 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:29.478690 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:30.478867 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:30.478829 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:30.479327 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:30.478829 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:30.479327 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:30.478982 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:30.479327 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:30.479103 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:31.479286 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:31.479255 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:31.479748 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:31.479360 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:32.478980 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:32.478945 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:32.479144 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:32.478987 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:32.479144 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:32.479061 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:32.479144 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:32.479136 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:33.479844 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.479435 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:33.480729 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:33.479947 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:33.578426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.578398 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:13:33.578843 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.578814 2564 generic.go:356] "Generic (PLEG): container finished" podID="827e2908-fba6-44a1-bfb4-6b22753d0eee" containerID="c9975e0770043efe86cce5a295c38fc180ea0e43fcc29a430cd2e7d00709175b" exitCode=1 Jul 16 00:13:33.578943 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.578895 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" event={"ID":"827e2908-fba6-44a1-bfb4-6b22753d0eee","Type":"ContainerStarted","Data":"d83f81989c0baf13453d72c07e9c11b44c84335c5e793205e7e10415403842a7"} Jul 16 00:13:33.579010 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.578968 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" event={"ID":"827e2908-fba6-44a1-bfb4-6b22753d0eee","Type":"ContainerStarted","Data":"5081a487b1fc56c4be74a6350b7463d16c2cd5b6e752ca5c6b1204d051c7b790"} Jul 16 00:13:33.579062 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.579017 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" event={"ID":"827e2908-fba6-44a1-bfb4-6b22753d0eee","Type":"ContainerStarted","Data":"5a30563552719dfa5d2bdf9176c68c9dd4b96a8c57186422f13ee740a145305b"} Jul 16 00:13:33.579115 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.579069 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" event={"ID":"827e2908-fba6-44a1-bfb4-6b22753d0eee","Type":"ContainerDied","Data":"c9975e0770043efe86cce5a295c38fc180ea0e43fcc29a430cd2e7d00709175b"} Jul 16 00:13:33.579168 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.579121 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" event={"ID":"827e2908-fba6-44a1-bfb4-6b22753d0eee","Type":"ContainerStarted","Data":"6859a5bce42d1549bcef1f25de9d998fa999347c98fd0d1f53514371ef59bc19"} Jul 16 00:13:33.580660 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.580632 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sf9f6" event={"ID":"63cc176e-fbac-4a75-877f-db6742663c92","Type":"ContainerStarted","Data":"d20ca818de4ff1faef97cfaa7d20e744fd215acc7e45cc76ac8d1f4fb4425f01"} Jul 16 00:13:33.582054 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.582018 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mcr99" event={"ID":"d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c","Type":"ContainerStarted","Data":"196f6d9d3662126a844e86587c93ed7723cb5e44ef7b609db5e3c3149a705b95"} Jul 16 00:13:33.583644 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.583612 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-btt64" event={"ID":"8a49dd8c-7e28-416a-9e61-102f27d5c7df","Type":"ContainerStarted","Data":"fe5c60de6043519fa6c6e971edef709f96b15fa6b2d363b878210c017a3ecf90"} Jul 16 00:13:33.585292 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.585258 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mk792" event={"ID":"375a83e3-da87-4562-b809-09a2f4fa9f22","Type":"ContainerStarted","Data":"a0bd76d6c186da68a6a2a77bfb6ae502538039bd3346ce812a2692d84d31cfd1"} Jul 16 00:13:33.586918 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.586890 2564 generic.go:356] "Generic (PLEG): container finished" podID="4089f83d-6157-48c3-8ed0-aa5bf7f01f7b" containerID="60230a7cbfc772772be56f4b3d0528da2902c29e24ce3310e2a6d56b2ddddaac" exitCode=0 Jul 16 00:13:33.587018 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.586969 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rltww" event={"ID":"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b","Type":"ContainerDied","Data":"60230a7cbfc772772be56f4b3d0528da2902c29e24ce3310e2a6d56b2ddddaac"} Jul 16 00:13:33.588654 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.588629 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" event={"ID":"4a0f53d2-f79b-4267-843a-4b58096fc425","Type":"ContainerStarted","Data":"79da33b6ae92d0cf801f02a62aca2e27bd221be67af3e190f224ea79dbe30b49"} Jul 16 00:13:33.590375 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.590351 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r5xhf" event={"ID":"6487ac13-1c94-4d7f-917b-f2f14892e58d","Type":"ContainerStarted","Data":"320c127e397496abb7dc725eafbfb4f57b078d77d33a0a9fc403ebb673eaa112"} Jul 16 00:13:33.595460 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.595414 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-224.ec2.internal" podStartSLOduration=21.595395114 podStartE2EDuration="21.595395114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:13:16.554100334 +0000 UTC m=+5.622040810" watchObservedRunningTime="2026-07-16 00:13:33.595395114 +0000 UTC m=+22.663335590" Jul 16 00:13:33.595781 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.595746 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sf9f6" podStartSLOduration=4.044750691 podStartE2EDuration="22.595736222s" totalImagesPullingTime="18.550985531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:13:33.594825602 +0000 UTC m=+22.662766078" watchObservedRunningTime="2026-07-16 00:13:33.595736222 +0000 UTC m=+22.663676700" Jul 16 00:13:33.609487 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.609359 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-btt64" podStartSLOduration=12.033209307 podStartE2EDuration="22.609336653s" totalImagesPullingTime="10.576127346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:13:33.608814524 +0000 UTC m=+22.676755000" watchObservedRunningTime="2026-07-16 00:13:33.609336653 +0000 UTC m=+22.677277153" Jul 16 00:13:33.645496 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.645440 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mcr99" podStartSLOduration=4.095895292 podStartE2EDuration="22.645424151s" totalImagesPullingTime="18.549528859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:13:33.645009221 +0000 UTC m=+22.712949695" watchObservedRunningTime="2026-07-16 00:13:33.645424151 +0000 UTC m=+22.713364625" Jul 16 00:13:33.662038 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.661981 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-r5xhf" podStartSLOduration=4.058950064 podStartE2EDuration="22.661961828s" totalImagesPullingTime="18.603011764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:13:33.661584766 +0000 UTC m=+22.729525235" watchObservedRunningTime="2026-07-16 00:13:33.661961828 +0000 UTC m=+22.729902318" Jul 16 00:13:33.678561 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:33.678230 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mk792" podStartSLOduration=4.122796361 podStartE2EDuration="22.678208871s" totalImagesPullingTime="18.55541251s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:13:33.676996651 +0000 UTC m=+22.744937126" watchObservedRunningTime="2026-07-16 00:13:33.678208871 +0000 UTC m=+22.746149348" Jul 16 00:13:34.235279 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:34.235238 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 00:13:34.372413 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:34.372298 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T00:13:34.235268325Z","UUID":"5a51d890-0acc-414b-93fb-c9d44f83da62","Handler":null,"Name":"","Endpoint":""} Jul 16 00:13:34.374050 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:34.374030 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 00:13:34.374153 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:34.374075 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 00:13:34.478722 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:34.478678 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:34.478948 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:34.478842 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:34.478948 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:34.478902 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:34.479072 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:34.479019 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:34.594096 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:34.594056 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w647c" event={"ID":"eb681aac-22c3-4d6c-b238-79d5ee473549","Type":"ContainerStarted","Data":"30c152bc83873524cb9601a5c7e666fcac46abc69384725a94fc50c79835212f"} Jul 16 00:13:34.596265 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:34.596237 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" event={"ID":"4a0f53d2-f79b-4267-843a-4b58096fc425","Type":"ContainerStarted","Data":"8c41079092f86d5180e7d7b3214bea3ed8e640d6d98865fae9b0bb384e3fb760"} Jul 16 00:13:34.599555 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:34.599513 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:13:34.599992 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:34.599966 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" event={"ID":"827e2908-fba6-44a1-bfb4-6b22753d0eee","Type":"ContainerStarted","Data":"d011bc21dc724704a1ef4e052eb7eb1d41bf671bfd2cd0d1482050c2c596e001"} Jul 16 00:13:34.609458 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:34.609411 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-w647c" podStartSLOduration=5.063938781 podStartE2EDuration="23.609397083s" totalImagesPullingTime="18.545458302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:13:34.608801291 +0000 UTC m=+23.676741760" watchObservedRunningTime="2026-07-16 00:13:34.609397083 +0000 UTC m=+23.677337558" Jul 16 00:13:35.479578 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:35.479550 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:35.479769 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:35.479692 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:35.604964 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:35.604920 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" event={"ID":"4a0f53d2-f79b-4267-843a-4b58096fc425","Type":"ContainerStarted","Data":"05ab3c53f7a6a0f1d908466d82511aa70022addf2e121d9133124ea029b75bda"} Jul 16 00:13:35.622882 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:35.622782 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcsf8" podStartSLOduration=3.8774271110000003 podStartE2EDuration="24.62276098s" totalImagesPullingTime="20.745333869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:13:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:13:35.622723612 +0000 UTC m=+24.690664088" watchObservedRunningTime="2026-07-16 00:13:35.62276098 +0000 UTC m=+24.690701452" Jul 16 00:13:36.479040 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:36.479002 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:36.479321 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:36.479055 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:36.479321 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:36.479154 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:36.479446 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:36.479346 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:36.610658 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:36.610633 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:13:36.611115 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:36.611044 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" event={"ID":"827e2908-fba6-44a1-bfb4-6b22753d0eee","Type":"ContainerStarted","Data":"31379bbc43ce4a07e0624895221e894e4204169f39f52371af208181700d641e"} Jul 16 00:13:36.881885 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:36.881847 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-btt64" Jul 16 00:13:36.882628 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:36.882596 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-btt64" Jul 16 00:13:37.479603 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:37.479360 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:37.479787 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:37.479713 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:38.478727 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:38.478692 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:38.479491 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:38.478709 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:38.479491 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:38.478839 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:38.479491 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:38.478871 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:38.618598 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:38.618571 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:13:38.618953 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:38.618933 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" event={"ID":"827e2908-fba6-44a1-bfb4-6b22753d0eee","Type":"ContainerStarted","Data":"ba58d08083663ce840f756032d0479b5c851bc7144e12cb7b36f5a8ec742af08"} Jul 16 00:13:38.619190 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:38.619175 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:38.619372 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:38.619355 2564 scope.go:117] "RemoveContainer" containerID="c9975e0770043efe86cce5a295c38fc180ea0e43fcc29a430cd2e7d00709175b" Jul 16 00:13:38.620903 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:38.620882 2564 generic.go:356] "Generic (PLEG): container finished" podID="4089f83d-6157-48c3-8ed0-aa5bf7f01f7b" containerID="61e62fbb3bc3d7acd689fd28b1f6eb7f7dc04933f8c68d356d47a87a9c46f6bd" exitCode=0 Jul 16 00:13:38.620994 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:38.620908 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rltww" event={"ID":"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b","Type":"ContainerDied","Data":"61e62fbb3bc3d7acd689fd28b1f6eb7f7dc04933f8c68d356d47a87a9c46f6bd"} Jul 16 00:13:38.635974 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:38.635953 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:39.478982 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:39.478769 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:39.479460 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:39.479048 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:39.627019 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:39.626994 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:13:39.627400 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:39.627361 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" event={"ID":"827e2908-fba6-44a1-bfb4-6b22753d0eee","Type":"ContainerStarted","Data":"ac698e6f249dd98c5a2702c61155a36978e9bcebc5b32e85e24c17dafbff13c0"} Jul 16 00:13:39.627748 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:39.627723 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:39.627860 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:39.627821 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:39.643785 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:39.643752 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:13:39.655554 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:39.655493 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" podStartSLOduration=10.027252043 podStartE2EDuration="28.655479463s" totalImagesPullingTime="18.62822742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:13:39.653668078 +0000 UTC m=+28.721608553" watchObservedRunningTime="2026-07-16 00:13:39.655479463 +0000 UTC m=+28.723419937" Jul 16 00:13:39.906050 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:39.906019 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wzszc"] Jul 16 00:13:39.906222 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:39.906137 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:39.906281 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:39.906222 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:39.909356 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:39.909090 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xl7gl"] Jul 16 00:13:39.909356 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:39.909212 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:39.909356 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:39.909316 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:39.909949 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:39.909924 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l725h"] Jul 16 00:13:39.910217 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:39.910200 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:39.910455 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:39.910432 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:40.058973 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:40.058939 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-btt64" Jul 16 00:13:40.059134 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:40.059071 2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 16 00:13:40.059608 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:40.059591 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-btt64" Jul 16 00:13:40.633084 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:40.633050 2564 generic.go:356] "Generic (PLEG): container finished" podID="4089f83d-6157-48c3-8ed0-aa5bf7f01f7b" containerID="2028a311e13ccb33927daa162c146ce9331f76464276a1624f0ae1622457f427" exitCode=0 Jul 16 00:13:40.633566 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:40.633128 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rltww" event={"ID":"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b","Type":"ContainerDied","Data":"2028a311e13ccb33927daa162c146ce9331f76464276a1624f0ae1622457f427"} Jul 16 00:13:41.479916 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:41.479881 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:41.480116 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:41.479986 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:41.480116 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:41.480017 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:41.480116 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:41.480038 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:41.480269 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:41.480160 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:41.480269 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:41.480246 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:42.641735 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:42.641527 2564 generic.go:356] "Generic (PLEG): container finished" podID="4089f83d-6157-48c3-8ed0-aa5bf7f01f7b" containerID="5ce848a53e6a0fcec4d9cae8c5622d5195caa4c7a6b59e5b7cadf88f76029092" exitCode=0 Jul 16 00:13:42.642164 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:42.641562 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rltww" event={"ID":"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b","Type":"ContainerDied","Data":"5ce848a53e6a0fcec4d9cae8c5622d5195caa4c7a6b59e5b7cadf88f76029092"} Jul 16 00:13:43.478836 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:43.478808 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:43.479029 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:43.478805 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:43.479029 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:43.478925 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:43.479029 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:43.479013 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:43.479197 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:43.478805 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:43.479197 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:43.479104 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:45.060811 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.060773 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tpnb\" (UniqueName: \"kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb\") pod \"network-check-target-wzszc\" (UID: \"3c4e248d-a32f-48ee-a338-55864ccbfa19\") " pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:45.061463 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.060839 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:45.061463 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:45.060943 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 00:13:45.061463 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:45.060962 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 00:13:45.061463 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:45.060966 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:45.061463 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:45.060972 2564 projected.go:196] Error preparing data for projected volume kube-api-access-9tpnb for pod openshift-network-diagnostics/network-check-target-wzszc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:45.061463 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:45.061027 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret podName:868d72f4-c70a-47ab-842d-5d22e5586970 nodeName:}" failed. No retries permitted until 2026-07-16 00:14:17.061010231 +0000 UTC m=+66.128950685 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret") pod "global-pull-secret-syncer-xl7gl" (UID: "868d72f4-c70a-47ab-842d-5d22e5586970") : object "kube-system"/"original-pull-secret" not registered Jul 16 00:13:45.061463 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:45.061042 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb podName:3c4e248d-a32f-48ee-a338-55864ccbfa19 nodeName:}" failed. No retries permitted until 2026-07-16 00:14:17.061036347 +0000 UTC m=+66.128976800 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9tpnb" (UniqueName: "kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb") pod "network-check-target-wzszc" (UID: "3c4e248d-a32f-48ee-a338-55864ccbfa19") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 00:13:45.161905 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.161871 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:45.162078 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:45.162026 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:45.162133 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:45.162094 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs podName:5f9ca78d-1eb6-45bc-bf9a-01b646191c67 nodeName:}" failed. No retries permitted until 2026-07-16 00:14:17.162079264 +0000 UTC m=+66.230019720 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs") pod "network-metrics-daemon-l725h" (UID: "5f9ca78d-1eb6-45bc-bf9a-01b646191c67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 00:13:45.479016 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.478979 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:45.479016 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.479019 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:45.479244 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:45.479116 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xl7gl" podUID="868d72f4-c70a-47ab-842d-5d22e5586970" Jul 16 00:13:45.479244 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:45.479199 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:13:45.479244 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.479216 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:45.479400 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:45.479308 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzszc" podUID="3c4e248d-a32f-48ee-a338-55864ccbfa19" Jul 16 00:13:45.764169 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.764137 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-224.ec2.internal" event="NodeReady" Jul 16 00:13:45.764339 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.764294 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 00:13:45.808251 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.808207 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m5p7c"] Jul 16 00:13:45.833224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.833189 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-k8wkl"] Jul 16 00:13:45.833374 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.833333 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m5p7c" Jul 16 00:13:45.835829 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.835803 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 00:13:45.835976 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.835881 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 00:13:45.835976 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.835892 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kj65f\"" Jul 16 00:13:45.835976 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.835937 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 00:13:45.846904 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.846873 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m5p7c"] Jul 16 00:13:45.847040 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.846918 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k8wkl"] Jul 16 00:13:45.847102 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.847033 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:45.849379 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.849360 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 00:13:45.849558 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.849433 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 00:13:45.849558 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.849470 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-djqcn\"" Jul 16 00:13:45.968066 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.968015 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-tmp-dir\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:45.968066 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.968069 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:45.968288 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.968097 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-config-volume\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:45.968288 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.968112 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d5xc\" (UniqueName: \"kubernetes.io/projected/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-kube-api-access-7d5xc\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:45.968288 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.968222 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce5278f9-b320-4971-92ef-a33f0804bb39-cert\") pod \"ingress-canary-m5p7c\" (UID: \"ce5278f9-b320-4971-92ef-a33f0804bb39\") " pod="openshift-ingress-canary/ingress-canary-m5p7c" Jul 16 00:13:45.968406 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:45.968289 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xsf2\" (UniqueName: \"kubernetes.io/projected/ce5278f9-b320-4971-92ef-a33f0804bb39-kube-api-access-4xsf2\") pod \"ingress-canary-m5p7c\" (UID: \"ce5278f9-b320-4971-92ef-a33f0804bb39\") " pod="openshift-ingress-canary/ingress-canary-m5p7c" Jul 16 00:13:46.069062 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.068974 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xsf2\" (UniqueName: \"kubernetes.io/projected/ce5278f9-b320-4971-92ef-a33f0804bb39-kube-api-access-4xsf2\") pod \"ingress-canary-m5p7c\" (UID: \"ce5278f9-b320-4971-92ef-a33f0804bb39\") " pod="openshift-ingress-canary/ingress-canary-m5p7c" Jul 16 00:13:46.069062 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.069016 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-tmp-dir\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:46.069062 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.069048 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:46.069814 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.069078 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-config-volume\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:46.069814 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.069104 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d5xc\" (UniqueName: \"kubernetes.io/projected/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-kube-api-access-7d5xc\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:46.069814 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.069161 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce5278f9-b320-4971-92ef-a33f0804bb39-cert\") pod \"ingress-canary-m5p7c\" (UID: \"ce5278f9-b320-4971-92ef-a33f0804bb39\") " pod="openshift-ingress-canary/ingress-canary-m5p7c" Jul 16 00:13:46.069814 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:46.069219 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:13:46.069814 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:46.069293 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls podName:f5645e23-1659-4fa0-b4f8-7cc6e7f31d96 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:46.569271269 +0000 UTC m=+35.637211731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls") pod "dns-default-k8wkl" (UID: "f5645e23-1659-4fa0-b4f8-7cc6e7f31d96") : secret "dns-default-metrics-tls" not found Jul 16 00:13:46.069814 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.069397 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-tmp-dir\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:46.070118 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.069851 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-config-volume\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:46.074033 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.074009 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce5278f9-b320-4971-92ef-a33f0804bb39-cert\") pod \"ingress-canary-m5p7c\" (UID: \"ce5278f9-b320-4971-92ef-a33f0804bb39\") " pod="openshift-ingress-canary/ingress-canary-m5p7c" Jul 16 00:13:46.077954 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.077933 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xsf2\" (UniqueName: \"kubernetes.io/projected/ce5278f9-b320-4971-92ef-a33f0804bb39-kube-api-access-4xsf2\") pod \"ingress-canary-m5p7c\" (UID: \"ce5278f9-b320-4971-92ef-a33f0804bb39\") " pod="openshift-ingress-canary/ingress-canary-m5p7c" Jul 16 00:13:46.078075 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.078016 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d5xc\" (UniqueName: \"kubernetes.io/projected/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-kube-api-access-7d5xc\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:46.144627 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.144591 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m5p7c" Jul 16 00:13:46.573912 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.573869 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:46.574112 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:46.574033 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:13:46.574180 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:46.574126 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls podName:f5645e23-1659-4fa0-b4f8-7cc6e7f31d96 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:47.574103368 +0000 UTC m=+36.642043820 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls") pod "dns-default-k8wkl" (UID: "f5645e23-1659-4fa0-b4f8-7cc6e7f31d96") : secret "dns-default-metrics-tls" not found Jul 16 00:13:46.981671 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:46.981478 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m5p7c"] Jul 16 00:13:47.053494 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:13:47.053457 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce5278f9_b320_4971_92ef_a33f0804bb39.slice/crio-c745f2bd80f453eefaf67aa67d7505f2c672b8c2af4415a34b02a3e5fb9eaa6a WatchSource:0}: Error finding container c745f2bd80f453eefaf67aa67d7505f2c672b8c2af4415a34b02a3e5fb9eaa6a: Status 404 returned error can't find the container with id c745f2bd80f453eefaf67aa67d7505f2c672b8c2af4415a34b02a3e5fb9eaa6a Jul 16 00:13:47.478841 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:47.478756 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:13:47.478841 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:47.478775 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:13:47.479471 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:47.479043 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:13:47.481529 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:47.481509 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 00:13:47.481529 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:47.481509 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sl9p7\"" Jul 16 00:13:47.482414 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:47.482399 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 00:13:47.482508 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:47.482438 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 00:13:47.482508 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:47.482474 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ct92c\"" Jul 16 00:13:47.482645 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:47.482474 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 00:13:47.581770 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:47.581735 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:47.581971 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:47.581841 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:13:47.581971 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:47.581895 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls podName:f5645e23-1659-4fa0-b4f8-7cc6e7f31d96 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:49.581882651 +0000 UTC m=+38.649823104 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls") pod "dns-default-k8wkl" (UID: "f5645e23-1659-4fa0-b4f8-7cc6e7f31d96") : secret "dns-default-metrics-tls" not found Jul 16 00:13:47.653671 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:47.653636 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m5p7c" event={"ID":"ce5278f9-b320-4971-92ef-a33f0804bb39","Type":"ContainerStarted","Data":"c745f2bd80f453eefaf67aa67d7505f2c672b8c2af4415a34b02a3e5fb9eaa6a"} Jul 16 00:13:47.656280 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:47.656251 2564 generic.go:356] "Generic (PLEG): container finished" podID="4089f83d-6157-48c3-8ed0-aa5bf7f01f7b" containerID="2d81e7db3d452ce40d0888d6c1a4c8baa910d64966eb2cd297c8efe98b9e6031" exitCode=0 Jul 16 00:13:47.656408 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:47.656300 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rltww" event={"ID":"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b","Type":"ContainerDied","Data":"2d81e7db3d452ce40d0888d6c1a4c8baa910d64966eb2cd297c8efe98b9e6031"} Jul 16 00:13:48.663082 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:48.663050 2564 generic.go:356] "Generic (PLEG): container finished" podID="4089f83d-6157-48c3-8ed0-aa5bf7f01f7b" containerID="d9192b8fd5426227deb9ef9111fde8268dbb80089cc9897b7f56dc93c854b830" exitCode=0 Jul 16 00:13:48.663633 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:48.663116 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rltww" event={"ID":"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b","Type":"ContainerDied","Data":"d9192b8fd5426227deb9ef9111fde8268dbb80089cc9897b7f56dc93c854b830"} Jul 16 00:13:49.599156 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:49.599110 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:49.599329 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:49.599241 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:13:49.599329 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:49.599307 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls podName:f5645e23-1659-4fa0-b4f8-7cc6e7f31d96 nodeName:}" failed. No retries permitted until 2026-07-16 00:13:53.599285338 +0000 UTC m=+42.667225814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls") pod "dns-default-k8wkl" (UID: "f5645e23-1659-4fa0-b4f8-7cc6e7f31d96") : secret "dns-default-metrics-tls" not found Jul 16 00:13:49.669820 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:49.669589 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rltww" event={"ID":"4089f83d-6157-48c3-8ed0-aa5bf7f01f7b","Type":"ContainerStarted","Data":"11357a59b934ddfa18bcbd11c47746f735a263f8e992e587d372390fe644da0f"} Jul 16 00:13:49.695284 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:49.695226 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rltww" podStartSLOduration=9.143192706 podStartE2EDuration="38.695203939s" totalImagesPullingTime="29.552011233s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:13:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:13:49.693306729 +0000 UTC m=+38.761247244" watchObservedRunningTime="2026-07-16 00:13:49.695203939 +0000 UTC m=+38.763144414" Jul 16 00:13:50.673438 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:50.673402 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m5p7c" event={"ID":"ce5278f9-b320-4971-92ef-a33f0804bb39","Type":"ContainerStarted","Data":"3dba7e8b8651911a35d5ce823c21b89fc5863ab9da58603832e1f384f307b4d7"} Jul 16 00:13:50.688232 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:50.688183 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m5p7c" podStartSLOduration=2.87283044 podStartE2EDuration="5.688163783s" totalImagesPullingTime="2.815333343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:13:50.687804318 +0000 UTC m=+39.755744794" watchObservedRunningTime="2026-07-16 00:13:50.688163783 +0000 UTC m=+39.756104257" Jul 16 00:13:53.631321 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:13:53.631283 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:13:53.631734 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:53.631402 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:13:53.631734 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:13:53.631452 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls podName:f5645e23-1659-4fa0-b4f8-7cc6e7f31d96 nodeName:}" failed. No retries permitted until 2026-07-16 00:14:01.631439808 +0000 UTC m=+50.699380262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls") pod "dns-default-k8wkl" (UID: "f5645e23-1659-4fa0-b4f8-7cc6e7f31d96") : secret "dns-default-metrics-tls" not found Jul 16 00:14:01.686547 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:01.686490 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:14:01.687065 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:14:01.686626 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:14:01.687065 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:14:01.686697 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls podName:f5645e23-1659-4fa0-b4f8-7cc6e7f31d96 nodeName:}" failed. No retries permitted until 2026-07-16 00:14:17.686681835 +0000 UTC m=+66.754622287 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls") pod "dns-default-k8wkl" (UID: "f5645e23-1659-4fa0-b4f8-7cc6e7f31d96") : secret "dns-default-metrics-tls" not found Jul 16 00:14:09.106040 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.106006 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn"] Jul 16 00:14:09.108702 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.108678 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn" Jul 16 00:14:09.109790 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.109766 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f"] Jul 16 00:14:09.111325 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.111299 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:14:09.112443 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.112425 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-vfdxz\"" Jul 16 00:14:09.112720 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.112703 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 16 00:14:09.112805 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.112726 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 16 00:14:09.112805 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.112772 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 16 00:14:09.112915 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.112814 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 16 00:14:09.113740 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.113727 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 16 00:14:09.119270 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.119249 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn"] Jul 16 00:14:09.122654 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.122633 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f"] Jul 16 00:14:09.132891 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.132870 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qktnx\" (UniqueName: \"kubernetes.io/projected/9fde11cc-6799-4ae0-9f94-919197b350c6-kube-api-access-qktnx\") pod \"klusterlet-addon-workmgr-79d59c66cd-qbt9f\" (UID: \"9fde11cc-6799-4ae0-9f94-919197b350c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:14:09.132975 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.132901 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz56w\" (UniqueName: \"kubernetes.io/projected/a5967a2e-a29f-44f0-96eb-2e9d72f11eb1-kube-api-access-fz56w\") pod \"managed-serviceaccount-addon-agent-7449cc6666-xqpgn\" (UID: \"a5967a2e-a29f-44f0-96eb-2e9d72f11eb1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn" Jul 16 00:14:09.132975 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.132949 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a5967a2e-a29f-44f0-96eb-2e9d72f11eb1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7449cc6666-xqpgn\" (UID: \"a5967a2e-a29f-44f0-96eb-2e9d72f11eb1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn" Jul 16 00:14:09.133080 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.132977 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9fde11cc-6799-4ae0-9f94-919197b350c6-klusterlet-config\") pod \"klusterlet-addon-workmgr-79d59c66cd-qbt9f\" (UID: \"9fde11cc-6799-4ae0-9f94-919197b350c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:14:09.133080 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.132995 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fde11cc-6799-4ae0-9f94-919197b350c6-tmp\") pod \"klusterlet-addon-workmgr-79d59c66cd-qbt9f\" (UID: \"9fde11cc-6799-4ae0-9f94-919197b350c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:14:09.233294 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.233259 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9fde11cc-6799-4ae0-9f94-919197b350c6-klusterlet-config\") pod \"klusterlet-addon-workmgr-79d59c66cd-qbt9f\" (UID: \"9fde11cc-6799-4ae0-9f94-919197b350c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:14:09.233294 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.233299 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fde11cc-6799-4ae0-9f94-919197b350c6-tmp\") pod \"klusterlet-addon-workmgr-79d59c66cd-qbt9f\" (UID: \"9fde11cc-6799-4ae0-9f94-919197b350c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:14:09.233568 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.233356 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qktnx\" (UniqueName: \"kubernetes.io/projected/9fde11cc-6799-4ae0-9f94-919197b350c6-kube-api-access-qktnx\") pod \"klusterlet-addon-workmgr-79d59c66cd-qbt9f\" (UID: \"9fde11cc-6799-4ae0-9f94-919197b350c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:14:09.233568 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.233373 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz56w\" (UniqueName: \"kubernetes.io/projected/a5967a2e-a29f-44f0-96eb-2e9d72f11eb1-kube-api-access-fz56w\") pod \"managed-serviceaccount-addon-agent-7449cc6666-xqpgn\" (UID: \"a5967a2e-a29f-44f0-96eb-2e9d72f11eb1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn" Jul 16 00:14:09.233568 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.233400 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a5967a2e-a29f-44f0-96eb-2e9d72f11eb1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7449cc6666-xqpgn\" (UID: \"a5967a2e-a29f-44f0-96eb-2e9d72f11eb1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn" Jul 16 00:14:09.233794 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.233775 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fde11cc-6799-4ae0-9f94-919197b350c6-tmp\") pod \"klusterlet-addon-workmgr-79d59c66cd-qbt9f\" (UID: \"9fde11cc-6799-4ae0-9f94-919197b350c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:14:09.237332 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.237305 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9fde11cc-6799-4ae0-9f94-919197b350c6-klusterlet-config\") pod \"klusterlet-addon-workmgr-79d59c66cd-qbt9f\" (UID: \"9fde11cc-6799-4ae0-9f94-919197b350c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:14:09.237442 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.237342 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a5967a2e-a29f-44f0-96eb-2e9d72f11eb1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7449cc6666-xqpgn\" (UID: \"a5967a2e-a29f-44f0-96eb-2e9d72f11eb1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn" Jul 16 00:14:09.241636 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.241616 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qktnx\" (UniqueName: \"kubernetes.io/projected/9fde11cc-6799-4ae0-9f94-919197b350c6-kube-api-access-qktnx\") pod \"klusterlet-addon-workmgr-79d59c66cd-qbt9f\" (UID: \"9fde11cc-6799-4ae0-9f94-919197b350c6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:14:09.241733 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.241644 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz56w\" (UniqueName: \"kubernetes.io/projected/a5967a2e-a29f-44f0-96eb-2e9d72f11eb1-kube-api-access-fz56w\") pod \"managed-serviceaccount-addon-agent-7449cc6666-xqpgn\" (UID: \"a5967a2e-a29f-44f0-96eb-2e9d72f11eb1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn" Jul 16 00:14:09.426519 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.426421 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn" Jul 16 00:14:09.432644 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.432625 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:14:09.556377 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.556343 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn"] Jul 16 00:14:09.559859 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:14:09.559826 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5967a2e_a29f_44f0_96eb_2e9d72f11eb1.slice/crio-78353fcfbb472466e3818998607d41ff4c6724b986c245ca01ccfb432028dacf WatchSource:0}: Error finding container 78353fcfbb472466e3818998607d41ff4c6724b986c245ca01ccfb432028dacf: Status 404 returned error can't find the container with id 78353fcfbb472466e3818998607d41ff4c6724b986c245ca01ccfb432028dacf Jul 16 00:14:09.573242 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.573181 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f"] Jul 16 00:14:09.576436 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:14:09.576409 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fde11cc_6799_4ae0_9f94_919197b350c6.slice/crio-bed856473ebe4f7f5e5217be0bca18e7851195e2175bbb830b45731f4a8fdaa4 WatchSource:0}: Error finding container bed856473ebe4f7f5e5217be0bca18e7851195e2175bbb830b45731f4a8fdaa4: Status 404 returned error can't find the container with id bed856473ebe4f7f5e5217be0bca18e7851195e2175bbb830b45731f4a8fdaa4 Jul 16 00:14:09.720299 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.720216 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" event={"ID":"9fde11cc-6799-4ae0-9f94-919197b350c6","Type":"ContainerStarted","Data":"bed856473ebe4f7f5e5217be0bca18e7851195e2175bbb830b45731f4a8fdaa4"} Jul 16 00:14:09.721238 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:09.721211 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn" event={"ID":"a5967a2e-a29f-44f0-96eb-2e9d72f11eb1","Type":"ContainerStarted","Data":"78353fcfbb472466e3818998607d41ff4c6724b986c245ca01ccfb432028dacf"} Jul 16 00:14:11.646660 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:11.646632 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qrwnm" Jul 16 00:14:12.732674 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:12.732579 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn" event={"ID":"a5967a2e-a29f-44f0-96eb-2e9d72f11eb1","Type":"ContainerStarted","Data":"7549673159f64b59c6aab20490c203143658a0ac95581d38738a818887bb262e"} Jul 16 00:14:12.746783 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:12.746706 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn" podStartSLOduration=0.976752024 podStartE2EDuration="3.746685318s" totalImagesPullingTime="2.769933294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:14:12.746629699 +0000 UTC m=+61.814570179" watchObservedRunningTime="2026-07-16 00:14:12.746685318 +0000 UTC m=+61.814625797" Jul 16 00:14:14.738651 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:14.738611 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" event={"ID":"9fde11cc-6799-4ae0-9f94-919197b350c6","Type":"ContainerStarted","Data":"11e61adb3baa0fd587a57efd37f98a243ba469d5dddf969d5c17a783c466293a"} Jul 16 00:14:14.739033 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:14.738854 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:14:14.740388 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:14.740370 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:14:14.754558 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:14.754374 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" podStartSLOduration=1.522003966 podStartE2EDuration="5.754356614s" totalImagesPullingTime="4.232352648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:14:14.753900607 +0000 UTC m=+63.821841084" watchObservedRunningTime="2026-07-16 00:14:14.754356614 +0000 UTC m=+63.822297095" Jul 16 00:14:17.097654 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.097610 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tpnb\" (UniqueName: \"kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb\") pod \"network-check-target-wzszc\" (UID: \"3c4e248d-a32f-48ee-a338-55864ccbfa19\") " pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:14:17.098095 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.097668 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:14:17.100127 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.100104 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 00:14:17.100183 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.100104 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 00:14:17.109871 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.109842 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 00:14:17.110256 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.110234 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/868d72f4-c70a-47ab-842d-5d22e5586970-original-pull-secret\") pod \"global-pull-secret-syncer-xl7gl\" (UID: \"868d72f4-c70a-47ab-842d-5d22e5586970\") " pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:14:17.121089 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.121062 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tpnb\" (UniqueName: \"kubernetes.io/projected/3c4e248d-a32f-48ee-a338-55864ccbfa19-kube-api-access-9tpnb\") pod \"network-check-target-wzszc\" (UID: \"3c4e248d-a32f-48ee-a338-55864ccbfa19\") " pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:14:17.191260 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.191234 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sl9p7\"" Jul 16 00:14:17.198292 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.198267 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:14:17.198582 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.198567 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xl7gl" Jul 16 00:14:17.199170 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.199157 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:14:17.200230 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.200214 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 00:14:17.209253 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:14:17.209201 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 00:14:17.209371 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:14:17.209338 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs podName:5f9ca78d-1eb6-45bc-bf9a-01b646191c67 nodeName:}" failed. No retries permitted until 2026-07-16 00:15:21.209311369 +0000 UTC m=+130.277251845 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs") pod "network-metrics-daemon-l725h" (UID: "5f9ca78d-1eb6-45bc-bf9a-01b646191c67") : secret "metrics-daemon-secret" not found Jul 16 00:14:17.331800 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.331771 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xl7gl"] Jul 16 00:14:17.334952 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:14:17.334924 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod868d72f4_c70a_47ab_842d_5d22e5586970.slice/crio-c4de5f6f4bcf7f1078424b662d8c8496de24c8e371dc0df249d67f139c53aea1 WatchSource:0}: Error finding container c4de5f6f4bcf7f1078424b662d8c8496de24c8e371dc0df249d67f139c53aea1: Status 404 returned error can't find the container with id c4de5f6f4bcf7f1078424b662d8c8496de24c8e371dc0df249d67f139c53aea1 Jul 16 00:14:17.344089 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.344065 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wzszc"] Jul 16 00:14:17.348574 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:14:17.348509 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c4e248d_a32f_48ee_a338_55864ccbfa19.slice/crio-3f082cbbf043da2a1e6e1f220e53addf672814aefaafda79e392b0a7eda95623 WatchSource:0}: Error finding container 3f082cbbf043da2a1e6e1f220e53addf672814aefaafda79e392b0a7eda95623: Status 404 returned error can't find the container with id 3f082cbbf043da2a1e6e1f220e53addf672814aefaafda79e392b0a7eda95623 Jul 16 00:14:17.702140 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.702054 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:14:17.702301 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:14:17.702235 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:14:17.702365 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:14:17.702314 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls podName:f5645e23-1659-4fa0-b4f8-7cc6e7f31d96 nodeName:}" failed. No retries permitted until 2026-07-16 00:14:49.702292097 +0000 UTC m=+98.770232560 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls") pod "dns-default-k8wkl" (UID: "f5645e23-1659-4fa0-b4f8-7cc6e7f31d96") : secret "dns-default-metrics-tls" not found Jul 16 00:14:17.748606 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.748563 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wzszc" event={"ID":"3c4e248d-a32f-48ee-a338-55864ccbfa19","Type":"ContainerStarted","Data":"3f082cbbf043da2a1e6e1f220e53addf672814aefaafda79e392b0a7eda95623"} Jul 16 00:14:17.749909 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:17.749880 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xl7gl" event={"ID":"868d72f4-c70a-47ab-842d-5d22e5586970","Type":"ContainerStarted","Data":"c4de5f6f4bcf7f1078424b662d8c8496de24c8e371dc0df249d67f139c53aea1"} Jul 16 00:14:22.766989 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:22.766952 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wzszc" event={"ID":"3c4e248d-a32f-48ee-a338-55864ccbfa19","Type":"ContainerStarted","Data":"a5b9ceaefbbf2b790f53c3cab00647ef82f3aadf0291ab46103e04a598f342a3"} Jul 16 00:14:22.767436 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:22.767045 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:14:22.768737 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:22.768707 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xl7gl" event={"ID":"868d72f4-c70a-47ab-842d-5d22e5586970","Type":"ContainerStarted","Data":"a39f9cfed4b0e7f34d1b48e9f35b82346e193824bdba01c1396fc9b656889e2c"} Jul 16 00:14:22.781690 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:22.781643 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wzszc" podStartSLOduration=67.170086986 podStartE2EDuration="1m11.7816254s" totalImagesPullingTime="4.611538414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:14:22.780893406 +0000 UTC m=+71.848833880" watchObservedRunningTime="2026-07-16 00:14:22.7816254 +0000 UTC m=+71.849565876" Jul 16 00:14:49.727224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:49.727159 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:14:49.727679 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:14:49.727303 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:14:49.727679 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:14:49.727374 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls podName:f5645e23-1659-4fa0-b4f8-7cc6e7f31d96 nodeName:}" failed. No retries permitted until 2026-07-16 00:15:53.727358613 +0000 UTC m=+162.795299070 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls") pod "dns-default-k8wkl" (UID: "f5645e23-1659-4fa0-b4f8-7cc6e7f31d96") : secret "dns-default-metrics-tls" not found Jul 16 00:14:53.774412 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:53.774380 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wzszc" Jul 16 00:14:53.794689 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:14:53.794634 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xl7gl" podStartSLOduration=97.82169197 podStartE2EDuration="1m42.794601974s" totalImagesPullingTime="4.972910004s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:14:22.795233984 +0000 UTC m=+71.863174460" watchObservedRunningTime="2026-07-16 00:14:53.794601974 +0000 UTC m=+102.862542430" Jul 16 00:15:21.243046 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:15:21.242993 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:15:21.243586 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:15:21.243143 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 00:15:21.243586 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:15:21.243221 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs podName:5f9ca78d-1eb6-45bc-bf9a-01b646191c67 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:23.24320546 +0000 UTC m=+252.311145913 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs") pod "network-metrics-daemon-l725h" (UID: "5f9ca78d-1eb6-45bc-bf9a-01b646191c67") : secret "metrics-daemon-secret" not found Jul 16 00:15:48.856638 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:15:48.856589 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-k8wkl" podUID="f5645e23-1659-4fa0-b4f8-7cc6e7f31d96" Jul 16 00:15:50.494691 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:15:50.494645 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-l725h" podUID="5f9ca78d-1eb6-45bc-bf9a-01b646191c67" Jul 16 00:15:53.776268 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:15:53.776234 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:15:53.776689 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:15:53.776381 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 00:15:53.776689 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:15:53.776444 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls podName:f5645e23-1659-4fa0-b4f8-7cc6e7f31d96 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:55.776428781 +0000 UTC m=+284.844369237 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls") pod "dns-default-k8wkl" (UID: "f5645e23-1659-4fa0-b4f8-7cc6e7f31d96") : secret "dns-default-metrics-tls" not found Jul 16 00:16:02.479100 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:16:02.479028 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:16:02.479429 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:16:02.479126 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k8wkl" Jul 16 00:16:13.081346 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:16:13.081310 2564 generic.go:356] "Generic (PLEG): container finished" podID="a5967a2e-a29f-44f0-96eb-2e9d72f11eb1" containerID="7549673159f64b59c6aab20490c203143658a0ac95581d38738a818887bb262e" exitCode=255 Jul 16 00:16:13.081787 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:16:13.081383 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn" event={"ID":"a5967a2e-a29f-44f0-96eb-2e9d72f11eb1","Type":"ContainerDied","Data":"7549673159f64b59c6aab20490c203143658a0ac95581d38738a818887bb262e"} Jul 16 00:16:13.086971 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:16:13.086952 2564 scope.go:117] "RemoveContainer" containerID="7549673159f64b59c6aab20490c203143658a0ac95581d38738a818887bb262e" Jul 16 00:16:14.088119 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:16:14.088082 2564 generic.go:356] "Generic (PLEG): container finished" podID="9fde11cc-6799-4ae0-9f94-919197b350c6" containerID="11e61adb3baa0fd587a57efd37f98a243ba469d5dddf969d5c17a783c466293a" exitCode=1 Jul 16 00:16:14.088509 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:16:14.088157 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" event={"ID":"9fde11cc-6799-4ae0-9f94-919197b350c6","Type":"ContainerDied","Data":"11e61adb3baa0fd587a57efd37f98a243ba469d5dddf969d5c17a783c466293a"} Jul 16 00:16:14.088584 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:16:14.088514 2564 scope.go:117] "RemoveContainer" containerID="11e61adb3baa0fd587a57efd37f98a243ba469d5dddf969d5c17a783c466293a" Jul 16 00:16:14.089948 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:16:14.089920 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7449cc6666-xqpgn" event={"ID":"a5967a2e-a29f-44f0-96eb-2e9d72f11eb1","Type":"ContainerStarted","Data":"4a9f410ede0218cefe44f91360a33bb517e41f4d19ae74c4afdfa2e73e88db13"} Jul 16 00:16:14.739529 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:16:14.739489 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:16:15.094279 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:16:15.094241 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" event={"ID":"9fde11cc-6799-4ae0-9f94-919197b350c6","Type":"ContainerStarted","Data":"639e0ca03caedf8e6d68c8e0876376707c9ea437a4f5479ccdaea461fb68c766"} Jul 16 00:16:15.094683 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:16:15.094440 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:16:15.095040 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:16:15.095022 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79d59c66cd-qbt9f" Jul 16 00:17:00.138938 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.138908 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j"] Jul 16 00:17:00.141086 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.141065 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" Jul 16 00:17:00.143304 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.143285 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qb2dp\"" Jul 16 00:17:00.143445 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.143421 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 16 00:17:00.144377 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.144362 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 16 00:17:00.144452 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.144369 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 16 00:17:00.155876 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.155857 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j"] Jul 16 00:17:00.215931 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.215883 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpmvd\" (UniqueName: \"kubernetes.io/projected/72c1137b-eff5-4f0f-9405-50d590792bb0-kube-api-access-kpmvd\") pod \"cluster-samples-operator-65db858894-nrq6j\" (UID: \"72c1137b-eff5-4f0f-9405-50d590792bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" Jul 16 00:17:00.216134 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.216033 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nrq6j\" (UID: \"72c1137b-eff5-4f0f-9405-50d590792bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" Jul 16 00:17:00.231370 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.231343 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9"] Jul 16 00:17:00.233069 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.233055 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-nsg4s"] Jul 16 00:17:00.233200 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.233185 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" Jul 16 00:17:00.234729 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.234712 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.238566 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.238526 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 16 00:17:00.238674 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.238646 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 16 00:17:00.238955 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.238749 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 16 00:17:00.238955 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.238776 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 16 00:17:00.238955 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.238752 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 16 00:17:00.238955 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.238792 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 16 00:17:00.238955 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.238833 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 16 00:17:00.238955 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.238854 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 16 00:17:00.239266 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.239062 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-kc4kc\"" Jul 16 00:17:00.239675 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.239604 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-87cff8cbd-fj9jz"] Jul 16 00:17:00.239675 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.239632 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-pwjh8\"" Jul 16 00:17:00.242475 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.241479 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.245988 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.245966 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 00:17:00.246085 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.246006 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cws9t\"" Jul 16 00:17:00.246142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.246089 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 00:17:00.246198 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.246189 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 00:17:00.248980 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.248961 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 16 00:17:00.250838 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.250820 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 00:17:00.256910 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.256893 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9"] Jul 16 00:17:00.257987 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.257968 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-nsg4s"] Jul 16 00:17:00.260069 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.260048 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-87cff8cbd-fj9jz"] Jul 16 00:17:00.316426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316395 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-bm2b9\" (UID: \"6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" Jul 16 00:17:00.316426 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316428 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89e7200c-a92a-4b5b-aead-a8279280af08-serving-cert\") pod \"console-operator-7866497494-nsg4s\" (UID: \"89e7200c-a92a-4b5b-aead-a8279280af08\") " pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.316717 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316455 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nrq6j\" (UID: \"72c1137b-eff5-4f0f-9405-50d590792bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" Jul 16 00:17:00.316717 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316471 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/28871d19-a84d-41f4-8b5a-35e934555f75-registry-certificates\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.316717 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316559 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/28871d19-a84d-41f4-8b5a-35e934555f75-image-registry-private-configuration\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.316717 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:00.316592 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 00:17:00.316717 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316614 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5q8s\" (UniqueName: \"kubernetes.io/projected/6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5-kube-api-access-w5q8s\") pod \"kube-storage-version-migrator-operator-799cb7f958-bm2b9\" (UID: \"6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" Jul 16 00:17:00.316717 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316646 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89e7200c-a92a-4b5b-aead-a8279280af08-trusted-ca\") pod \"console-operator-7866497494-nsg4s\" (UID: \"89e7200c-a92a-4b5b-aead-a8279280af08\") " pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.316717 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316673 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-bound-sa-token\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.316717 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316709 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/28871d19-a84d-41f4-8b5a-35e934555f75-ca-trust-extracted\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.316980 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:00.316738 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls podName:72c1137b-eff5-4f0f-9405-50d590792bb0 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:00.816717352 +0000 UTC m=+229.884657809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls") pod "cluster-samples-operator-65db858894-nrq6j" (UID: "72c1137b-eff5-4f0f-9405-50d590792bb0") : secret "samples-operator-tls" not found Jul 16 00:17:00.316980 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316785 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpmvd\" (UniqueName: \"kubernetes.io/projected/72c1137b-eff5-4f0f-9405-50d590792bb0-kube-api-access-kpmvd\") pod \"cluster-samples-operator-65db858894-nrq6j\" (UID: \"72c1137b-eff5-4f0f-9405-50d590792bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" Jul 16 00:17:00.316980 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316829 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwh4b\" (UniqueName: \"kubernetes.io/projected/89e7200c-a92a-4b5b-aead-a8279280af08-kube-api-access-hwh4b\") pod \"console-operator-7866497494-nsg4s\" (UID: \"89e7200c-a92a-4b5b-aead-a8279280af08\") " pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.316980 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316847 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/28871d19-a84d-41f4-8b5a-35e934555f75-trusted-ca\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.316980 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316867 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-bm2b9\" (UID: \"6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" Jul 16 00:17:00.316980 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316898 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92j4q\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-kube-api-access-92j4q\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.316980 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316938 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/28871d19-a84d-41f4-8b5a-35e934555f75-installation-pull-secrets\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.316980 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.316969 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.317240 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.317016 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89e7200c-a92a-4b5b-aead-a8279280af08-config\") pod \"console-operator-7866497494-nsg4s\" (UID: \"89e7200c-a92a-4b5b-aead-a8279280af08\") " pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.327486 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.327463 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpmvd\" (UniqueName: \"kubernetes.io/projected/72c1137b-eff5-4f0f-9405-50d590792bb0-kube-api-access-kpmvd\") pod \"cluster-samples-operator-65db858894-nrq6j\" (UID: \"72c1137b-eff5-4f0f-9405-50d590792bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" Jul 16 00:17:00.417851 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.417757 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-bm2b9\" (UID: \"6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" Jul 16 00:17:00.417851 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.417794 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89e7200c-a92a-4b5b-aead-a8279280af08-serving-cert\") pod \"console-operator-7866497494-nsg4s\" (UID: \"89e7200c-a92a-4b5b-aead-a8279280af08\") " pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.417851 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.417826 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/28871d19-a84d-41f4-8b5a-35e934555f75-registry-certificates\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.418102 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.417942 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/28871d19-a84d-41f4-8b5a-35e934555f75-image-registry-private-configuration\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.418102 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.417994 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5q8s\" (UniqueName: \"kubernetes.io/projected/6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5-kube-api-access-w5q8s\") pod \"kube-storage-version-migrator-operator-799cb7f958-bm2b9\" (UID: \"6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" Jul 16 00:17:00.418102 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.418020 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89e7200c-a92a-4b5b-aead-a8279280af08-trusted-ca\") pod \"console-operator-7866497494-nsg4s\" (UID: \"89e7200c-a92a-4b5b-aead-a8279280af08\") " pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.418102 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.418045 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-bound-sa-token\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.418102 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.418083 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/28871d19-a84d-41f4-8b5a-35e934555f75-ca-trust-extracted\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.418353 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.418139 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwh4b\" (UniqueName: \"kubernetes.io/projected/89e7200c-a92a-4b5b-aead-a8279280af08-kube-api-access-hwh4b\") pod \"console-operator-7866497494-nsg4s\" (UID: \"89e7200c-a92a-4b5b-aead-a8279280af08\") " pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.418353 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.418165 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/28871d19-a84d-41f4-8b5a-35e934555f75-trusted-ca\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.418353 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.418205 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-bm2b9\" (UID: \"6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" Jul 16 00:17:00.418353 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.418232 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92j4q\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-kube-api-access-92j4q\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.418353 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.418256 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/28871d19-a84d-41f4-8b5a-35e934555f75-installation-pull-secrets\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.418353 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.418283 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.418353 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.418329 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89e7200c-a92a-4b5b-aead-a8279280af08-config\") pod \"console-operator-7866497494-nsg4s\" (UID: \"89e7200c-a92a-4b5b-aead-a8279280af08\") " pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.418710 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.418375 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/28871d19-a84d-41f4-8b5a-35e934555f75-registry-certificates\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.418978 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.418954 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-bm2b9\" (UID: \"6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" Jul 16 00:17:00.419056 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.418979 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89e7200c-a92a-4b5b-aead-a8279280af08-trusted-ca\") pod \"console-operator-7866497494-nsg4s\" (UID: \"89e7200c-a92a-4b5b-aead-a8279280af08\") " pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.419109 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:00.419073 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 00:17:00.419109 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.419078 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89e7200c-a92a-4b5b-aead-a8279280af08-config\") pod \"console-operator-7866497494-nsg4s\" (UID: \"89e7200c-a92a-4b5b-aead-a8279280af08\") " pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.419109 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:00.419089 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87cff8cbd-fj9jz: secret "image-registry-tls" not found Jul 16 00:17:00.419245 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:00.419170 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls podName:28871d19-a84d-41f4-8b5a-35e934555f75 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:00.919150662 +0000 UTC m=+229.987091114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls") pod "image-registry-87cff8cbd-fj9jz" (UID: "28871d19-a84d-41f4-8b5a-35e934555f75") : secret "image-registry-tls" not found Jul 16 00:17:00.419245 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.419176 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/28871d19-a84d-41f4-8b5a-35e934555f75-trusted-ca\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.419381 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.419362 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/28871d19-a84d-41f4-8b5a-35e934555f75-ca-trust-extracted\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.420460 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.420427 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-bm2b9\" (UID: \"6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" Jul 16 00:17:00.420599 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.420515 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89e7200c-a92a-4b5b-aead-a8279280af08-serving-cert\") pod \"console-operator-7866497494-nsg4s\" (UID: \"89e7200c-a92a-4b5b-aead-a8279280af08\") " pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.420599 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.420564 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/28871d19-a84d-41f4-8b5a-35e934555f75-image-registry-private-configuration\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.421299 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.421279 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/28871d19-a84d-41f4-8b5a-35e934555f75-installation-pull-secrets\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.426164 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.426142 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5q8s\" (UniqueName: \"kubernetes.io/projected/6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5-kube-api-access-w5q8s\") pod \"kube-storage-version-migrator-operator-799cb7f958-bm2b9\" (UID: \"6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" Jul 16 00:17:00.427288 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.427266 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92j4q\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-kube-api-access-92j4q\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.427384 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.427329 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwh4b\" (UniqueName: \"kubernetes.io/projected/89e7200c-a92a-4b5b-aead-a8279280af08-kube-api-access-hwh4b\") pod \"console-operator-7866497494-nsg4s\" (UID: \"89e7200c-a92a-4b5b-aead-a8279280af08\") " pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.427686 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.427670 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-bound-sa-token\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.543954 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.543881 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" Jul 16 00:17:00.553774 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.553749 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:00.675681 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.675651 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9"] Jul 16 00:17:00.678130 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:17:00.678103 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b2fa53b_3900_4a5c_bf5f_f5a4cd8a35b5.slice/crio-6d24b9e2339465ce145069aaed8b21b7fa2bfd5d194c77edb73378a71132ef6a WatchSource:0}: Error finding container 6d24b9e2339465ce145069aaed8b21b7fa2bfd5d194c77edb73378a71132ef6a: Status 404 returned error can't find the container with id 6d24b9e2339465ce145069aaed8b21b7fa2bfd5d194c77edb73378a71132ef6a Jul 16 00:17:00.694019 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.693994 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-nsg4s"] Jul 16 00:17:00.696867 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:17:00.696847 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89e7200c_a92a_4b5b_aead_a8279280af08.slice/crio-18117809be207542a87746c2552b259b0fd41761f0d591c77983d8179c6a8f21 WatchSource:0}: Error finding container 18117809be207542a87746c2552b259b0fd41761f0d591c77983d8179c6a8f21: Status 404 returned error can't find the container with id 18117809be207542a87746c2552b259b0fd41761f0d591c77983d8179c6a8f21 Jul 16 00:17:00.822120 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.822079 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nrq6j\" (UID: \"72c1137b-eff5-4f0f-9405-50d590792bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" Jul 16 00:17:00.822288 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:00.822228 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 00:17:00.822334 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:00.822291 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls podName:72c1137b-eff5-4f0f-9405-50d590792bb0 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:01.822275628 +0000 UTC m=+230.890216080 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls") pod "cluster-samples-operator-65db858894-nrq6j" (UID: "72c1137b-eff5-4f0f-9405-50d590792bb0") : secret "samples-operator-tls" not found Jul 16 00:17:00.923457 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:00.923416 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:00.923620 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:00.923562 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 00:17:00.923620 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:00.923574 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87cff8cbd-fj9jz: secret "image-registry-tls" not found Jul 16 00:17:00.923711 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:00.923623 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls podName:28871d19-a84d-41f4-8b5a-35e934555f75 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:01.923610411 +0000 UTC m=+230.991550864 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls") pod "image-registry-87cff8cbd-fj9jz" (UID: "28871d19-a84d-41f4-8b5a-35e934555f75") : secret "image-registry-tls" not found Jul 16 00:17:01.228194 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:01.228156 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-nsg4s" event={"ID":"89e7200c-a92a-4b5b-aead-a8279280af08","Type":"ContainerStarted","Data":"18117809be207542a87746c2552b259b0fd41761f0d591c77983d8179c6a8f21"} Jul 16 00:17:01.229495 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:01.229466 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" event={"ID":"6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5","Type":"ContainerStarted","Data":"6d24b9e2339465ce145069aaed8b21b7fa2bfd5d194c77edb73378a71132ef6a"} Jul 16 00:17:01.830294 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:01.830260 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nrq6j\" (UID: \"72c1137b-eff5-4f0f-9405-50d590792bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" Jul 16 00:17:01.830474 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:01.830402 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 00:17:01.830547 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:01.830478 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls podName:72c1137b-eff5-4f0f-9405-50d590792bb0 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:03.830457085 +0000 UTC m=+232.898397540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls") pod "cluster-samples-operator-65db858894-nrq6j" (UID: "72c1137b-eff5-4f0f-9405-50d590792bb0") : secret "samples-operator-tls" not found Jul 16 00:17:01.931184 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:01.931145 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:01.931369 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:01.931313 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 00:17:01.931369 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:01.931336 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87cff8cbd-fj9jz: secret "image-registry-tls" not found Jul 16 00:17:01.931473 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:01.931404 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls podName:28871d19-a84d-41f4-8b5a-35e934555f75 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:03.931384162 +0000 UTC m=+232.999324615 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls") pod "image-registry-87cff8cbd-fj9jz" (UID: "28871d19-a84d-41f4-8b5a-35e934555f75") : secret "image-registry-tls" not found Jul 16 00:17:03.843233 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:03.843178 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nrq6j\" (UID: \"72c1137b-eff5-4f0f-9405-50d590792bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" Jul 16 00:17:03.843654 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:03.843344 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 00:17:03.843654 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:03.843410 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls podName:72c1137b-eff5-4f0f-9405-50d590792bb0 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:07.843396552 +0000 UTC m=+236.911337009 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls") pod "cluster-samples-operator-65db858894-nrq6j" (UID: "72c1137b-eff5-4f0f-9405-50d590792bb0") : secret "samples-operator-tls" not found Jul 16 00:17:03.944278 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:03.944238 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:03.944430 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:03.944360 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 00:17:03.944430 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:03.944372 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87cff8cbd-fj9jz: secret "image-registry-tls" not found Jul 16 00:17:03.944430 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:03.944418 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls podName:28871d19-a84d-41f4-8b5a-35e934555f75 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:07.944406472 +0000 UTC m=+237.012346925 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls") pod "image-registry-87cff8cbd-fj9jz" (UID: "28871d19-a84d-41f4-8b5a-35e934555f75") : secret "image-registry-tls" not found Jul 16 00:17:04.240507 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:04.240415 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" event={"ID":"6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5","Type":"ContainerStarted","Data":"0fc233acad64c4199b31c8fc0fba059a16adb9228049f78d0459331896a974de"} Jul 16 00:17:04.241897 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:04.241865 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/0.log" Jul 16 00:17:04.242033 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:04.241904 2564 generic.go:356] "Generic (PLEG): container finished" podID="89e7200c-a92a-4b5b-aead-a8279280af08" containerID="91bf54690e1ca337b06f9651298e6d59de25ae795cc24df4393eb78ca4042b5f" exitCode=255 Jul 16 00:17:04.242033 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:04.241951 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-nsg4s" event={"ID":"89e7200c-a92a-4b5b-aead-a8279280af08","Type":"ContainerDied","Data":"91bf54690e1ca337b06f9651298e6d59de25ae795cc24df4393eb78ca4042b5f"} Jul 16 00:17:04.242186 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:04.242171 2564 scope.go:117] "RemoveContainer" containerID="91bf54690e1ca337b06f9651298e6d59de25ae795cc24df4393eb78ca4042b5f" Jul 16 00:17:04.258226 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:04.258182 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" podStartSLOduration=1.453591457 podStartE2EDuration="4.258168045s" totalImagesPullingTime="2.804576588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:17:04.256713295 +0000 UTC m=+233.324653771" watchObservedRunningTime="2026-07-16 00:17:04.258168045 +0000 UTC m=+233.326108518" Jul 16 00:17:05.245942 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.245915 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:17:05.246327 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.246312 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/0.log" Jul 16 00:17:05.246366 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.246345 2564 generic.go:356] "Generic (PLEG): container finished" podID="89e7200c-a92a-4b5b-aead-a8279280af08" containerID="35f3de2247df4a9bed4cd78abb58c83c3f7d92bb2185189cdfebb6e1882b3dc8" exitCode=255 Jul 16 00:17:05.246461 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.246437 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-nsg4s" event={"ID":"89e7200c-a92a-4b5b-aead-a8279280af08","Type":"ContainerDied","Data":"35f3de2247df4a9bed4cd78abb58c83c3f7d92bb2185189cdfebb6e1882b3dc8"} Jul 16 00:17:05.246512 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.246491 2564 scope.go:117] "RemoveContainer" containerID="91bf54690e1ca337b06f9651298e6d59de25ae795cc24df4393eb78ca4042b5f" Jul 16 00:17:05.246729 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.246712 2564 scope.go:117] "RemoveContainer" containerID="35f3de2247df4a9bed4cd78abb58c83c3f7d92bb2185189cdfebb6e1882b3dc8" Jul 16 00:17:05.246917 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:05.246893 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-nsg4s_openshift-console-operator(89e7200c-a92a-4b5b-aead-a8279280af08)\"" pod="openshift-console-operator/console-operator-7866497494-nsg4s" podUID="89e7200c-a92a-4b5b-aead-a8279280af08" Jul 16 00:17:05.418886 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.418849 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m9xxt"] Jul 16 00:17:05.422293 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.422267 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:05.424837 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.424815 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 00:17:05.425215 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.425202 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hdqj5\"" Jul 16 00:17:05.425435 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.425420 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 00:17:05.425514 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.425453 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 00:17:05.425675 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.425657 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 00:17:05.434335 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.434313 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m9xxt"] Jul 16 00:17:05.557155 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.557126 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-data-volume\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:05.557317 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.557174 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-crio-socket\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:05.557317 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.557285 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:05.557414 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.557325 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2tb7\" (UniqueName: \"kubernetes.io/projected/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-api-access-k2tb7\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:05.557414 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.557379 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:05.658461 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.658406 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:05.658626 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.658486 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-data-volume\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:05.658626 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.658520 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-crio-socket\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:05.658626 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:05.658612 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 00:17:05.658749 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.658638 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:05.658749 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:05.658683 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls podName:acb05a14-8ed0-4a7c-be13-e995fb05ebc6 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:06.158663955 +0000 UTC m=+235.226604425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m9xxt" (UID: "acb05a14-8ed0-4a7c-be13-e995fb05ebc6") : secret "insights-runtime-extractor-tls" not found Jul 16 00:17:05.658749 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.658621 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-crio-socket\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:05.658749 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:05.658707 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 00:17:05.658749 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.658720 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2tb7\" (UniqueName: \"kubernetes.io/projected/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-api-access-k2tb7\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:05.658749 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:05.658752 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm podName:acb05a14-8ed0-4a7c-be13-e995fb05ebc6 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:06.158741241 +0000 UTC m=+235.226681693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m9xxt" (UID: "acb05a14-8ed0-4a7c-be13-e995fb05ebc6") : configmap "kube-rbac-proxy" not found Jul 16 00:17:05.659017 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.658857 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-data-volume\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:05.667062 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:05.667037 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2tb7\" (UniqueName: \"kubernetes.io/projected/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-api-access-k2tb7\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:06.162179 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:06.162142 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:06.162389 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:06.162202 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:06.162389 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:06.162273 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 00:17:06.162389 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:06.162298 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 00:17:06.162389 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:06.162346 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm podName:acb05a14-8ed0-4a7c-be13-e995fb05ebc6 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:07.162327529 +0000 UTC m=+236.230267985 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m9xxt" (UID: "acb05a14-8ed0-4a7c-be13-e995fb05ebc6") : configmap "kube-rbac-proxy" not found Jul 16 00:17:06.162389 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:06.162363 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls podName:acb05a14-8ed0-4a7c-be13-e995fb05ebc6 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:07.162355348 +0000 UTC m=+236.230295806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m9xxt" (UID: "acb05a14-8ed0-4a7c-be13-e995fb05ebc6") : secret "insights-runtime-extractor-tls" not found Jul 16 00:17:06.250549 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:06.250519 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:17:06.250907 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:06.250843 2564 scope.go:117] "RemoveContainer" containerID="35f3de2247df4a9bed4cd78abb58c83c3f7d92bb2185189cdfebb6e1882b3dc8" Jul 16 00:17:06.251002 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:06.250984 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-nsg4s_openshift-console-operator(89e7200c-a92a-4b5b-aead-a8279280af08)\"" pod="openshift-console-operator/console-operator-7866497494-nsg4s" podUID="89e7200c-a92a-4b5b-aead-a8279280af08" Jul 16 00:17:07.170072 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:07.170023 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:07.170234 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:07.170096 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:07.170234 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:07.170152 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 00:17:07.170234 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:07.170218 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 00:17:07.170351 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:07.170219 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm podName:acb05a14-8ed0-4a7c-be13-e995fb05ebc6 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:09.170203227 +0000 UTC m=+238.238143696 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m9xxt" (UID: "acb05a14-8ed0-4a7c-be13-e995fb05ebc6") : configmap "kube-rbac-proxy" not found Jul 16 00:17:07.170351 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:07.170264 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls podName:acb05a14-8ed0-4a7c-be13-e995fb05ebc6 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:09.170253045 +0000 UTC m=+238.238193497 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m9xxt" (UID: "acb05a14-8ed0-4a7c-be13-e995fb05ebc6") : secret "insights-runtime-extractor-tls" not found Jul 16 00:17:07.876784 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:07.876746 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nrq6j\" (UID: \"72c1137b-eff5-4f0f-9405-50d590792bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" Jul 16 00:17:07.877191 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:07.876866 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 00:17:07.877191 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:07.876920 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls podName:72c1137b-eff5-4f0f-9405-50d590792bb0 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:15.876906178 +0000 UTC m=+244.944846631 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls") pod "cluster-samples-operator-65db858894-nrq6j" (UID: "72c1137b-eff5-4f0f-9405-50d590792bb0") : secret "samples-operator-tls" not found Jul 16 00:17:07.977185 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:07.977149 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:07.977384 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:07.977299 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 00:17:07.977384 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:07.977316 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87cff8cbd-fj9jz: secret "image-registry-tls" not found Jul 16 00:17:07.977384 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:07.977376 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls podName:28871d19-a84d-41f4-8b5a-35e934555f75 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:15.977358363 +0000 UTC m=+245.045298815 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls") pod "image-registry-87cff8cbd-fj9jz" (UID: "28871d19-a84d-41f4-8b5a-35e934555f75") : secret "image-registry-tls" not found Jul 16 00:17:08.007399 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:08.007374 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mcr99_d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c/dns-node-resolver/0.log" Jul 16 00:17:09.187695 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:09.187661 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:09.188076 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:09.187719 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:09.188076 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:09.187779 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 00:17:09.188076 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:09.187837 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 00:17:09.188076 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:09.187850 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm podName:acb05a14-8ed0-4a7c-be13-e995fb05ebc6 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:13.187833782 +0000 UTC m=+242.255774235 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m9xxt" (UID: "acb05a14-8ed0-4a7c-be13-e995fb05ebc6") : configmap "kube-rbac-proxy" not found Jul 16 00:17:09.188076 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:09.187876 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls podName:acb05a14-8ed0-4a7c-be13-e995fb05ebc6 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:13.187866146 +0000 UTC m=+242.255806598 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m9xxt" (UID: "acb05a14-8ed0-4a7c-be13-e995fb05ebc6") : secret "insights-runtime-extractor-tls" not found Jul 16 00:17:09.405396 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:09.405371 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sf9f6_63cc176e-fbac-4a75-877f-db6742663c92/node-ca/0.log" Jul 16 00:17:10.004053 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:10.004024 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m5p7c_ce5278f9-b320-4971-92ef-a33f0804bb39/serve-healthcheck-canary/0.log" Jul 16 00:17:10.554643 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:10.554611 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:10.555004 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:10.554661 2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:10.555004 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:10.554938 2564 scope.go:117] "RemoveContainer" containerID="35f3de2247df4a9bed4cd78abb58c83c3f7d92bb2185189cdfebb6e1882b3dc8" Jul 16 00:17:10.555089 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:10.555071 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-nsg4s_openshift-console-operator(89e7200c-a92a-4b5b-aead-a8279280af08)\"" pod="openshift-console-operator/console-operator-7866497494-nsg4s" podUID="89e7200c-a92a-4b5b-aead-a8279280af08" Jul 16 00:17:10.806288 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:10.806213 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-bm2b9_6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5/kube-storage-version-migrator-operator/0.log" Jul 16 00:17:13.219459 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:13.219418 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:13.219847 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:13.219478 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:13.219847 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:13.219529 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 00:17:13.219847 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:13.219604 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 00:17:13.219847 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:13.219606 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm podName:acb05a14-8ed0-4a7c-be13-e995fb05ebc6 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:21.219591206 +0000 UTC m=+250.287531658 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m9xxt" (UID: "acb05a14-8ed0-4a7c-be13-e995fb05ebc6") : configmap "kube-rbac-proxy" not found Jul 16 00:17:13.219847 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:17:13.219652 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls podName:acb05a14-8ed0-4a7c-be13-e995fb05ebc6 nodeName:}" failed. No retries permitted until 2026-07-16 00:17:21.219640734 +0000 UTC m=+250.287581187 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m9xxt" (UID: "acb05a14-8ed0-4a7c-be13-e995fb05ebc6") : secret "insights-runtime-extractor-tls" not found Jul 16 00:17:15.942095 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:15.942040 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nrq6j\" (UID: \"72c1137b-eff5-4f0f-9405-50d590792bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" Jul 16 00:17:15.944392 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:15.944372 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/72c1137b-eff5-4f0f-9405-50d590792bb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nrq6j\" (UID: \"72c1137b-eff5-4f0f-9405-50d590792bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" Jul 16 00:17:16.043395 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:16.043341 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:16.045915 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:16.045882 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls\") pod \"image-registry-87cff8cbd-fj9jz\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:16.052417 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:16.052393 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qb2dp\"" Jul 16 00:17:16.060483 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:16.060460 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" Jul 16 00:17:16.161090 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:16.161062 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cws9t\"" Jul 16 00:17:16.168999 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:16.168962 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:16.181707 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:16.181664 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j"] Jul 16 00:17:16.283471 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:16.283440 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" event={"ID":"72c1137b-eff5-4f0f-9405-50d590792bb0","Type":"ContainerStarted","Data":"512e0a55c4e8115a190827169b04303ca8ee5b9c97073200cfabb13a71ab94b5"} Jul 16 00:17:16.293790 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:16.293768 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-87cff8cbd-fj9jz"] Jul 16 00:17:16.296149 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:17:16.296121 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28871d19_a84d_41f4_8b5a_35e934555f75.slice/crio-87573b0b83b4d55b495ad0598330d81a155d68a5a312626455a80ac9c8e914ad WatchSource:0}: Error finding container 87573b0b83b4d55b495ad0598330d81a155d68a5a312626455a80ac9c8e914ad: Status 404 returned error can't find the container with id 87573b0b83b4d55b495ad0598330d81a155d68a5a312626455a80ac9c8e914ad Jul 16 00:17:17.288195 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:17.288160 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" event={"ID":"28871d19-a84d-41f4-8b5a-35e934555f75","Type":"ContainerStarted","Data":"96017031768b14e23984182da75065c0b53e6060aad4a9bd1da3ca5b457a9a2a"} Jul 16 00:17:17.288695 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:17.288211 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" event={"ID":"28871d19-a84d-41f4-8b5a-35e934555f75","Type":"ContainerStarted","Data":"87573b0b83b4d55b495ad0598330d81a155d68a5a312626455a80ac9c8e914ad"} Jul 16 00:17:17.288695 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:17.288317 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:17.311253 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:17.311192 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" podStartSLOduration=17.311168172 podStartE2EDuration="17.311168172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:17:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:17:17.309808947 +0000 UTC m=+246.377749422" watchObservedRunningTime="2026-07-16 00:17:17.311168172 +0000 UTC m=+246.379108629" Jul 16 00:17:19.296573 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:19.296506 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" event={"ID":"72c1137b-eff5-4f0f-9405-50d590792bb0","Type":"ContainerStarted","Data":"ee4bcbdbeb7e5486b1f530df40a4b4a99383bc8d321342a38a9c52a9d0cf6cbe"} Jul 16 00:17:19.296963 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:19.296624 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" event={"ID":"72c1137b-eff5-4f0f-9405-50d590792bb0","Type":"ContainerStarted","Data":"4bfeb2677699351a527ab75e19b7d9d5970f38148889831ec4a6ea5b019fe82d"} Jul 16 00:17:19.313552 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:19.313495 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nrq6j" podStartSLOduration=17.257368541 podStartE2EDuration="19.313476829s" totalImagesPullingTime="2.056108288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:17:19.31298548 +0000 UTC m=+248.380925954" watchObservedRunningTime="2026-07-16 00:17:19.313476829 +0000 UTC m=+248.381417318" Jul 16 00:17:21.286216 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:21.286176 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:21.286715 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:21.286234 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:21.286870 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:21.286846 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:21.288544 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:21.288516 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acb05a14-8ed0-4a7c-be13-e995fb05ebc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m9xxt\" (UID: \"acb05a14-8ed0-4a7c-be13-e995fb05ebc6\") " pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:21.332986 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:21.332950 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hdqj5\"" Jul 16 00:17:21.341087 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:21.341052 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m9xxt" Jul 16 00:17:21.470681 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:21.470643 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m9xxt"] Jul 16 00:17:21.474546 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:17:21.474511 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacb05a14_8ed0_4a7c_be13_e995fb05ebc6.slice/crio-5992384b90491e60db9d750e960c6befc2758ae37c3e0913eb1d152f15428d81 WatchSource:0}: Error finding container 5992384b90491e60db9d750e960c6befc2758ae37c3e0913eb1d152f15428d81: Status 404 returned error can't find the container with id 5992384b90491e60db9d750e960c6befc2758ae37c3e0913eb1d152f15428d81 Jul 16 00:17:22.308242 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:22.308203 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m9xxt" event={"ID":"acb05a14-8ed0-4a7c-be13-e995fb05ebc6","Type":"ContainerStarted","Data":"5992384b90491e60db9d750e960c6befc2758ae37c3e0913eb1d152f15428d81"} Jul 16 00:17:23.303408 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:23.303361 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:17:23.306250 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:23.306217 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f9ca78d-1eb6-45bc-bf9a-01b646191c67-metrics-certs\") pod \"network-metrics-daemon-l725h\" (UID: \"5f9ca78d-1eb6-45bc-bf9a-01b646191c67\") " pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:17:23.482798 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:23.482764 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ct92c\"" Jul 16 00:17:23.491162 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:23.491125 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l725h" Jul 16 00:17:23.655326 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:23.655063 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l725h"] Jul 16 00:17:23.658277 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:17:23.658245 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f9ca78d_1eb6_45bc_bf9a_01b646191c67.slice/crio-6ab8d270bd503d7e99b578c4ecaa853b9932a7ccebee4d54ebcc1596bbd2cca4 WatchSource:0}: Error finding container 6ab8d270bd503d7e99b578c4ecaa853b9932a7ccebee4d54ebcc1596bbd2cca4: Status 404 returned error can't find the container with id 6ab8d270bd503d7e99b578c4ecaa853b9932a7ccebee4d54ebcc1596bbd2cca4 Jul 16 00:17:24.316983 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:24.316945 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l725h" event={"ID":"5f9ca78d-1eb6-45bc-bf9a-01b646191c67","Type":"ContainerStarted","Data":"6ab8d270bd503d7e99b578c4ecaa853b9932a7ccebee4d54ebcc1596bbd2cca4"} Jul 16 00:17:25.479718 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:25.479688 2564 scope.go:117] "RemoveContainer" containerID="35f3de2247df4a9bed4cd78abb58c83c3f7d92bb2185189cdfebb6e1882b3dc8" Jul 16 00:17:26.916905 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:26.916866 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-87cff8cbd-fj9jz"] Jul 16 00:17:27.214441 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.214358 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-6n2bd"] Jul 16 00:17:27.217599 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.217572 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-6n2bd" Jul 16 00:17:27.220142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.220118 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 00:17:27.220278 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.220208 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 00:17:27.220341 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.220269 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7hf7m\"" Jul 16 00:17:27.228919 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.228893 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-6n2bd"] Jul 16 00:17:27.235235 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.235212 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fkst\" (UniqueName: \"kubernetes.io/projected/1fb1a268-c9cb-4bb2-bcba-a33d1944ce2c-kube-api-access-9fkst\") pod \"downloads-68c9cbb497-6n2bd\" (UID: \"1fb1a268-c9cb-4bb2-bcba-a33d1944ce2c\") " pod="openshift-console/downloads-68c9cbb497-6n2bd" Jul 16 00:17:27.328555 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.328499 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m9xxt" event={"ID":"acb05a14-8ed0-4a7c-be13-e995fb05ebc6","Type":"ContainerStarted","Data":"7ab22f16d06c63fbd1ec88bb8bb1634ae24c5468988d939e65141e9eea0b4a70"} Jul 16 00:17:27.330157 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.330130 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l725h" event={"ID":"5f9ca78d-1eb6-45bc-bf9a-01b646191c67","Type":"ContainerStarted","Data":"33db8753b35cce2b96825a2751cc66ed70c2f453b130c88931e8569109ec00dd"} Jul 16 00:17:27.330287 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.330185 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l725h" event={"ID":"5f9ca78d-1eb6-45bc-bf9a-01b646191c67","Type":"ContainerStarted","Data":"db75f174b81918ab31315ca0c0e680d618f27a0152be0a5934277eb094363ccf"} Jul 16 00:17:27.332066 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.332047 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:17:27.332148 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.332096 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-nsg4s" event={"ID":"89e7200c-a92a-4b5b-aead-a8279280af08","Type":"ContainerStarted","Data":"b3e719af7d78c5a69e55b404c285687e07b4e2754a4b174dbc34029947d0d444"} Jul 16 00:17:27.332349 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.332324 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:27.336044 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.336022 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkst\" (UniqueName: \"kubernetes.io/projected/1fb1a268-c9cb-4bb2-bcba-a33d1944ce2c-kube-api-access-9fkst\") pod \"downloads-68c9cbb497-6n2bd\" (UID: \"1fb1a268-c9cb-4bb2-bcba-a33d1944ce2c\") " pod="openshift-console/downloads-68c9cbb497-6n2bd" Jul 16 00:17:27.337567 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.337527 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-nsg4s" Jul 16 00:17:27.346881 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.346858 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fkst\" (UniqueName: \"kubernetes.io/projected/1fb1a268-c9cb-4bb2-bcba-a33d1944ce2c-kube-api-access-9fkst\") pod \"downloads-68c9cbb497-6n2bd\" (UID: \"1fb1a268-c9cb-4bb2-bcba-a33d1944ce2c\") " pod="openshift-console/downloads-68c9cbb497-6n2bd" Jul 16 00:17:27.349784 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.349729 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l725h" podStartSLOduration=253.187366379 podStartE2EDuration="4m16.349716077s" totalImagesPullingTime="3.162349698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:17:27.34943565 +0000 UTC m=+256.417376138" watchObservedRunningTime="2026-07-16 00:17:27.349716077 +0000 UTC m=+256.417656551" Jul 16 00:17:27.369219 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.369164 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-nsg4s" podStartSLOduration=24.582033547 podStartE2EDuration="27.369130006s" totalImagesPullingTime="2.787096459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:17:27.368455674 +0000 UTC m=+256.436396150" watchObservedRunningTime="2026-07-16 00:17:27.369130006 +0000 UTC m=+256.437070481" Jul 16 00:17:27.526680 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.526621 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-6n2bd" Jul 16 00:17:27.700791 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:27.700578 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-6n2bd"] Jul 16 00:17:27.705381 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:17:27.705346 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fb1a268_c9cb_4bb2_bcba_a33d1944ce2c.slice/crio-293a3a017a0b826c70dc82866a6cb23ee3bf8b2da28685360e7844b6c8523524 WatchSource:0}: Error finding container 293a3a017a0b826c70dc82866a6cb23ee3bf8b2da28685360e7844b6c8523524: Status 404 returned error can't find the container with id 293a3a017a0b826c70dc82866a6cb23ee3bf8b2da28685360e7844b6c8523524 Jul 16 00:17:28.336450 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:28.336413 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-6n2bd" event={"ID":"1fb1a268-c9cb-4bb2-bcba-a33d1944ce2c","Type":"ContainerStarted","Data":"293a3a017a0b826c70dc82866a6cb23ee3bf8b2da28685360e7844b6c8523524"} Jul 16 00:17:31.349897 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:31.349846 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m9xxt" event={"ID":"acb05a14-8ed0-4a7c-be13-e995fb05ebc6","Type":"ContainerStarted","Data":"feec10486c84d2c57b0f7cee18c8c7f5067672854b01824c96f9bb7f8b0eb83f"} Jul 16 00:17:35.373993 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:35.373762 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m9xxt" event={"ID":"acb05a14-8ed0-4a7c-be13-e995fb05ebc6","Type":"ContainerStarted","Data":"54d82c12fa62680eb7aaca3e6f59f40d040d9e7fda72c34da65a4ae1be1c4bd2"} Jul 16 00:17:35.396489 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:35.396435 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m9xxt" podStartSLOduration=17.408894999 podStartE2EDuration="30.396419145s" totalImagesPullingTime="12.987524146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:17:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:17:35.395111349 +0000 UTC m=+264.463051837" watchObservedRunningTime="2026-07-16 00:17:35.396419145 +0000 UTC m=+264.464359621" Jul 16 00:17:36.909561 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.906739 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b"] Jul 16 00:17:36.927322 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.927283 2564 patch_prober.go:28] interesting pod/image-registry-87cff8cbd-fj9jz container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 00:17:36.927486 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.927344 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" podUID="28871d19-a84d-41f4-8b5a-35e934555f75" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 00:17:36.938215 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.938188 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-58d9q"] Jul 16 00:17:36.938390 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.938330 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:36.941140 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.940891 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 16 00:17:36.941140 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.940901 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 00:17:36.941140 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.940901 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 00:17:36.941140 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.941136 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 00:17:36.941461 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.941344 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4l44p\"" Jul 16 00:17:36.941461 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.941346 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 16 00:17:36.952190 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.952168 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9zq5h"] Jul 16 00:17:36.952327 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.952310 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:36.955235 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.955214 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 16 00:17:36.955235 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.955224 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 16 00:17:36.955570 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.955497 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-wqcnf\"" Jul 16 00:17:36.955641 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.955591 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 16 00:17:36.970188 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.970165 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b"] Jul 16 00:17:36.970309 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.970210 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-58d9q"] Jul 16 00:17:36.970367 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.970331 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:36.973013 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.972966 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 00:17:36.973402 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.973324 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 00:17:36.973627 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.973605 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 00:17:36.973741 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:36.973693 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2h5dm\"" Jul 16 00:17:37.021709 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.021667 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c9734b3-3d18-4827-aaa0-898ac71c02bb-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.021902 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.021716 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-accelerators-collector-config\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.021902 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.021749 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6rkj\" (UniqueName: \"kubernetes.io/projected/4c9734b3-3d18-4827-aaa0-898ac71c02bb-kube-api-access-t6rkj\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.021902 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.021776 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/705ef190-5007-4564-b4f2-8b714fe92eb0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-nbq5b\" (UID: \"705ef190-5007-4564-b4f2-8b714fe92eb0\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:37.021902 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.021815 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s66j9\" (UniqueName: \"kubernetes.io/projected/705ef190-5007-4564-b4f2-8b714fe92eb0-kube-api-access-s66j9\") pod \"openshift-state-metrics-84bd68b896-nbq5b\" (UID: \"705ef190-5007-4564-b4f2-8b714fe92eb0\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:37.021902 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.021839 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c9734b3-3d18-4827-aaa0-898ac71c02bb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.021902 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.021868 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/733af086-2bdf-4a4d-bec0-61bc4e461723-root\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.021902 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.021891 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.022276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.021921 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-wtmp\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.022276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.022008 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/705ef190-5007-4564-b4f2-8b714fe92eb0-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-nbq5b\" (UID: \"705ef190-5007-4564-b4f2-8b714fe92eb0\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:37.022276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.022036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4c9734b3-3d18-4827-aaa0-898ac71c02bb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.022276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.022065 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4c9734b3-3d18-4827-aaa0-898ac71c02bb-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.022276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.022089 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/705ef190-5007-4564-b4f2-8b714fe92eb0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-nbq5b\" (UID: \"705ef190-5007-4564-b4f2-8b714fe92eb0\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:37.022276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.022115 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/733af086-2bdf-4a4d-bec0-61bc4e461723-sys\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.022276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.022176 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/733af086-2bdf-4a4d-bec0-61bc4e461723-metrics-client-ca\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.022276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.022207 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-tls\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.022276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.022231 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n7nt\" (UniqueName: \"kubernetes.io/projected/733af086-2bdf-4a4d-bec0-61bc4e461723-kube-api-access-2n7nt\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.022609 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.022285 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-textfile\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.022609 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.022316 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c9734b3-3d18-4827-aaa0-898ac71c02bb-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.122892 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.122853 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-wtmp\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.123076 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.122917 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/705ef190-5007-4564-b4f2-8b714fe92eb0-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-nbq5b\" (UID: \"705ef190-5007-4564-b4f2-8b714fe92eb0\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:37.123076 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.122944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4c9734b3-3d18-4827-aaa0-898ac71c02bb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.123076 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.122993 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4c9734b3-3d18-4827-aaa0-898ac71c02bb-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.123076 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123016 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/705ef190-5007-4564-b4f2-8b714fe92eb0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-nbq5b\" (UID: \"705ef190-5007-4564-b4f2-8b714fe92eb0\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:37.123076 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123047 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-wtmp\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.123326 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123089 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/733af086-2bdf-4a4d-bec0-61bc4e461723-sys\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.123326 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123124 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/733af086-2bdf-4a4d-bec0-61bc4e461723-metrics-client-ca\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.123326 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123151 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-tls\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.123326 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123181 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n7nt\" (UniqueName: \"kubernetes.io/projected/733af086-2bdf-4a4d-bec0-61bc4e461723-kube-api-access-2n7nt\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.123326 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123195 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/733af086-2bdf-4a4d-bec0-61bc4e461723-sys\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.123326 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123222 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-textfile\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.123326 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123252 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c9734b3-3d18-4827-aaa0-898ac71c02bb-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.123326 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123299 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c9734b3-3d18-4827-aaa0-898ac71c02bb-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.123761 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123331 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-accelerators-collector-config\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.123761 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123366 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6rkj\" (UniqueName: \"kubernetes.io/projected/4c9734b3-3d18-4827-aaa0-898ac71c02bb-kube-api-access-t6rkj\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.123761 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123391 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/705ef190-5007-4564-b4f2-8b714fe92eb0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-nbq5b\" (UID: \"705ef190-5007-4564-b4f2-8b714fe92eb0\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:37.123761 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123415 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s66j9\" (UniqueName: \"kubernetes.io/projected/705ef190-5007-4564-b4f2-8b714fe92eb0-kube-api-access-s66j9\") pod \"openshift-state-metrics-84bd68b896-nbq5b\" (UID: \"705ef190-5007-4564-b4f2-8b714fe92eb0\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:37.123761 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123448 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c9734b3-3d18-4827-aaa0-898ac71c02bb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.123761 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123451 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4c9734b3-3d18-4827-aaa0-898ac71c02bb-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.123761 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123474 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/733af086-2bdf-4a4d-bec0-61bc4e461723-root\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.123761 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123499 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.124155 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123768 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/733af086-2bdf-4a4d-bec0-61bc4e461723-metrics-client-ca\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.124155 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123826 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/705ef190-5007-4564-b4f2-8b714fe92eb0-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-nbq5b\" (UID: \"705ef190-5007-4564-b4f2-8b714fe92eb0\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:37.124155 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.123846 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4c9734b3-3d18-4827-aaa0-898ac71c02bb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.124155 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.124001 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-accelerators-collector-config\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.124155 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.124020 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-textfile\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.124155 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.124082 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/733af086-2bdf-4a4d-bec0-61bc4e461723-root\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.124724 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.124675 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c9734b3-3d18-4827-aaa0-898ac71c02bb-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.126521 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.126474 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-tls\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.126944 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.126897 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/733af086-2bdf-4a4d-bec0-61bc4e461723-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.127042 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.126978 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c9734b3-3d18-4827-aaa0-898ac71c02bb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.127102 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.127049 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/705ef190-5007-4564-b4f2-8b714fe92eb0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-nbq5b\" (UID: \"705ef190-5007-4564-b4f2-8b714fe92eb0\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:37.127102 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.127053 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c9734b3-3d18-4827-aaa0-898ac71c02bb-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.127399 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.127381 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/705ef190-5007-4564-b4f2-8b714fe92eb0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-nbq5b\" (UID: \"705ef190-5007-4564-b4f2-8b714fe92eb0\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:37.131677 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.131652 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n7nt\" (UniqueName: \"kubernetes.io/projected/733af086-2bdf-4a4d-bec0-61bc4e461723-kube-api-access-2n7nt\") pod \"node-exporter-9zq5h\" (UID: \"733af086-2bdf-4a4d-bec0-61bc4e461723\") " pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.132077 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.132060 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6rkj\" (UniqueName: \"kubernetes.io/projected/4c9734b3-3d18-4827-aaa0-898ac71c02bb-kube-api-access-t6rkj\") pod \"kube-state-metrics-785778ff74-58d9q\" (UID: \"4c9734b3-3d18-4827-aaa0-898ac71c02bb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.132221 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.132198 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s66j9\" (UniqueName: \"kubernetes.io/projected/705ef190-5007-4564-b4f2-8b714fe92eb0-kube-api-access-s66j9\") pod \"openshift-state-metrics-84bd68b896-nbq5b\" (UID: \"705ef190-5007-4564-b4f2-8b714fe92eb0\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:37.249640 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.249562 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" Jul 16 00:17:37.262549 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.262506 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" Jul 16 00:17:37.281272 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.281245 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9zq5h" Jul 16 00:17:37.318983 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:17:37.318948 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733af086_2bdf_4a4d_bec0_61bc4e461723.slice/crio-e90d2a327ca2c0d7c4a2049340d458850fcb2fb1c7d07f2dab083a850dce69ec WatchSource:0}: Error finding container e90d2a327ca2c0d7c4a2049340d458850fcb2fb1c7d07f2dab083a850dce69ec: Status 404 returned error can't find the container with id e90d2a327ca2c0d7c4a2049340d458850fcb2fb1c7d07f2dab083a850dce69ec Jul 16 00:17:37.382317 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.382252 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9zq5h" event={"ID":"733af086-2bdf-4a4d-bec0-61bc4e461723","Type":"ContainerStarted","Data":"e90d2a327ca2c0d7c4a2049340d458850fcb2fb1c7d07f2dab083a850dce69ec"} Jul 16 00:17:37.429579 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.429519 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b"] Jul 16 00:17:37.432826 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:17:37.432788 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod705ef190_5007_4564_b4f2_8b714fe92eb0.slice/crio-5259657f5734e18dcbd0baf9741d6c2bd34580c6e57babdce3177fd9d9e61e04 WatchSource:0}: Error finding container 5259657f5734e18dcbd0baf9741d6c2bd34580c6e57babdce3177fd9d9e61e04: Status 404 returned error can't find the container with id 5259657f5734e18dcbd0baf9741d6c2bd34580c6e57babdce3177fd9d9e61e04 Jul 16 00:17:37.455361 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:37.455315 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-58d9q"] Jul 16 00:17:37.463749 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:17:37.463717 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c9734b3_3d18_4827_aaa0_898ac71c02bb.slice/crio-f177f9961103d388fc934ce66d34169e9fa4f6ff5e953a94d01dad3e56c0ad79 WatchSource:0}: Error finding container f177f9961103d388fc934ce66d34169e9fa4f6ff5e953a94d01dad3e56c0ad79: Status 404 returned error can't find the container with id f177f9961103d388fc934ce66d34169e9fa4f6ff5e953a94d01dad3e56c0ad79 Jul 16 00:17:38.389392 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:38.389346 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" event={"ID":"705ef190-5007-4564-b4f2-8b714fe92eb0","Type":"ContainerStarted","Data":"dea7d7d973088558330735ecd76b2b55418321efe720cf9f10f9eef4f0f16e69"} Jul 16 00:17:38.389960 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:38.389464 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" event={"ID":"705ef190-5007-4564-b4f2-8b714fe92eb0","Type":"ContainerStarted","Data":"123088dcb7ed2c7152073a91c0456783ce45494b74ddbbb3f68028b4f9a0b7fc"} Jul 16 00:17:38.389960 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:38.389498 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" event={"ID":"705ef190-5007-4564-b4f2-8b714fe92eb0","Type":"ContainerStarted","Data":"5259657f5734e18dcbd0baf9741d6c2bd34580c6e57babdce3177fd9d9e61e04"} Jul 16 00:17:38.390937 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:38.390867 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" event={"ID":"4c9734b3-3d18-4827-aaa0-898ac71c02bb","Type":"ContainerStarted","Data":"f177f9961103d388fc934ce66d34169e9fa4f6ff5e953a94d01dad3e56c0ad79"} Jul 16 00:17:39.396300 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:39.396210 2564 generic.go:356] "Generic (PLEG): container finished" podID="733af086-2bdf-4a4d-bec0-61bc4e461723" containerID="74ae67f177d020b1af971424413048e43dd8b93f01726f1a4861f8fbbcb0f8b6" exitCode=0 Jul 16 00:17:39.396300 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:39.396277 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9zq5h" event={"ID":"733af086-2bdf-4a4d-bec0-61bc4e461723","Type":"ContainerDied","Data":"74ae67f177d020b1af971424413048e43dd8b93f01726f1a4861f8fbbcb0f8b6"} Jul 16 00:17:43.124005 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.123970 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 00:17:43.129148 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.129122 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.131893 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.131866 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 16 00:17:43.132048 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.131944 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 16 00:17:43.132119 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.132100 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 16 00:17:43.132214 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.132131 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 16 00:17:43.132214 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.132195 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ic81su8cd3nd\"" Jul 16 00:17:43.132415 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.132396 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 16 00:17:43.132508 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.132443 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 16 00:17:43.132972 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.132950 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 16 00:17:43.133105 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.133049 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 16 00:17:43.133225 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.133179 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 16 00:17:43.133424 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.133409 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hqx9j\"" Jul 16 00:17:43.133499 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.133482 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 16 00:17:43.133594 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.133515 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 16 00:17:43.133660 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.133608 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 16 00:17:43.133660 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.133654 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 16 00:17:43.134271 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.134251 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 16 00:17:43.135256 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.135235 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 16 00:17:43.144220 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.144195 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 00:17:43.178409 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178367 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178611 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178432 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178611 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178464 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178611 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178482 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a48d61ed-d382-4b5f-9b59-bb34c9c20570-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178611 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178505 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178611 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178560 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178611 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178576 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-web-config\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178611 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178609 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178934 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178666 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178934 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178715 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a48d61ed-d382-4b5f-9b59-bb34c9c20570-config-out\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178934 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178735 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178934 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178759 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178934 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178812 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178934 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178837 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178934 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178860 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178934 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178890 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178934 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178912 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.178934 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178934 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-config\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.179322 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178975 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.179322 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.178998 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdqrd\" (UniqueName: \"kubernetes.io/projected/a48d61ed-d382-4b5f-9b59-bb34c9c20570-kube-api-access-cdqrd\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279436 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279397 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279630 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279454 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-web-config\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279630 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279486 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279630 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279523 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279630 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279578 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a48d61ed-d382-4b5f-9b59-bb34c9c20570-config-out\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279630 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279600 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279630 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279624 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279947 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279657 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279947 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279682 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279947 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279707 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279947 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279738 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279947 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279764 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279947 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279809 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-config\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279947 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279867 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279947 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279894 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdqrd\" (UniqueName: \"kubernetes.io/projected/a48d61ed-d382-4b5f-9b59-bb34c9c20570-kube-api-access-cdqrd\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.279947 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279931 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.280372 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.279982 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.280372 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.280018 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.280372 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.280051 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a48d61ed-d382-4b5f-9b59-bb34c9c20570-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.280372 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.280076 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.280806 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.280776 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.281029 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.280968 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.281170 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.281143 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.281401 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.281333 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.281492 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.281410 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.282985 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.282957 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.283435 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.283402 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.283726 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.283704 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.284353 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.284325 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.284741 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.284720 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.285075 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.285044 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.285640 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.285476 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.286114 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.285880 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-web-config\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.286114 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.286050 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.286264 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.286163 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-config\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.286264 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.286164 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a48d61ed-d382-4b5f-9b59-bb34c9c20570-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.286388 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.286363 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.286588 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.286566 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.287080 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.287036 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a48d61ed-d382-4b5f-9b59-bb34c9c20570-config-out\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.289241 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.289217 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdqrd\" (UniqueName: \"kubernetes.io/projected/a48d61ed-d382-4b5f-9b59-bb34c9c20570-kube-api-access-cdqrd\") pod \"prometheus-k8s-0\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:43.440346 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:43.440236 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:17:46.357202 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:46.357176 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 00:17:46.424978 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:46.424940 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" event={"ID":"4c9734b3-3d18-4827-aaa0-898ac71c02bb","Type":"ContainerStarted","Data":"59030fcf4c8f5dd7209f61387bfac887f9dc78fb8f7ec5a9b95c2be89e9f9763"} Jul 16 00:17:46.426805 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:46.426770 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9zq5h" event={"ID":"733af086-2bdf-4a4d-bec0-61bc4e461723","Type":"ContainerStarted","Data":"b28a892cb32bbf32193f397dd247e4fe353d8e3ff814fe4c97ce9a6794d3b0c7"} Jul 16 00:17:46.429073 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:46.429045 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" event={"ID":"705ef190-5007-4564-b4f2-8b714fe92eb0","Type":"ContainerStarted","Data":"c3fe0c5bf20a59b9a9e4598c21e4c7038e11e6b667b724c813c24d818f8c286b"} Jul 16 00:17:46.432513 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:17:46.432487 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda48d61ed_d382_4b5f_9b59_bb34c9c20570.slice/crio-de2bd1e1e58de4c6bb21a8bfa93c6857ed1f9afc1712671f8a5237556d3b89bb WatchSource:0}: Error finding container de2bd1e1e58de4c6bb21a8bfa93c6857ed1f9afc1712671f8a5237556d3b89bb: Status 404 returned error can't find the container with id de2bd1e1e58de4c6bb21a8bfa93c6857ed1f9afc1712671f8a5237556d3b89bb Jul 16 00:17:46.448924 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:46.448876 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nbq5b" podStartSLOduration=8.852431375 podStartE2EDuration="10.448852594s" totalImagesPullingTime="1.596421219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:17:46.447783721 +0000 UTC m=+275.515724199" watchObservedRunningTime="2026-07-16 00:17:46.448852594 +0000 UTC m=+275.516793075" Jul 16 00:17:46.921889 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:46.921852 2564 patch_prober.go:28] interesting pod/image-registry-87cff8cbd-fj9jz container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 00:17:46.922128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:46.921916 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" podUID="28871d19-a84d-41f4-8b5a-35e934555f75" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 00:17:47.438320 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:47.437518 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-6n2bd" event={"ID":"1fb1a268-c9cb-4bb2-bcba-a33d1944ce2c","Type":"ContainerStarted","Data":"141e858b299f05ad90a6adee968edd3a3f4dba1c8f1a8c33cca2a3bc3ce137c5"} Jul 16 00:17:47.438320 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:47.437774 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-6n2bd" Jul 16 00:17:47.442625 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:47.442570 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" event={"ID":"4c9734b3-3d18-4827-aaa0-898ac71c02bb","Type":"ContainerStarted","Data":"1286b5ed2fd92b42119ad2446b272f8a17d8f595e206219fb1ed4900a3d4612f"} Jul 16 00:17:47.443295 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:47.442933 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" event={"ID":"4c9734b3-3d18-4827-aaa0-898ac71c02bb","Type":"ContainerStarted","Data":"792a4804b714106777c83945a2f58b13c18998f21b28d752e4ce71e20859e786"} Jul 16 00:17:47.444941 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:47.444911 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerStarted","Data":"de2bd1e1e58de4c6bb21a8bfa93c6857ed1f9afc1712671f8a5237556d3b89bb"} Jul 16 00:17:47.449226 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:47.449198 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9zq5h" event={"ID":"733af086-2bdf-4a4d-bec0-61bc4e461723","Type":"ContainerStarted","Data":"eeadf332958fd26ceb7a847bd59d6e551dfb48cfce64f04167afaad7d9df2428"} Jul 16 00:17:47.452751 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:47.452726 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-6n2bd" Jul 16 00:17:47.455581 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:47.454804 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-6n2bd" podStartSLOduration=1.6583054659999998 podStartE2EDuration="20.454785804s" totalImagesPullingTime="18.796480338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:17:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:17:47.453423431 +0000 UTC m=+276.521363900" watchObservedRunningTime="2026-07-16 00:17:47.454785804 +0000 UTC m=+276.522726281" Jul 16 00:17:47.494933 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:47.494864 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9zq5h" podStartSLOduration=10.289665154 podStartE2EDuration="11.494842753s" totalImagesPullingTime="1.126564088s" totalInitContainerRuntime="78.613511ms" isStatefulPod=true podCreationTimestamp="2026-07-16 00:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:17:47.493559355 +0000 UTC m=+276.561499834" watchObservedRunningTime="2026-07-16 00:17:47.494842753 +0000 UTC m=+276.562783228" Jul 16 00:17:47.512574 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:47.511871 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-58d9q" podStartSLOduration=9.818900447 podStartE2EDuration="11.511847763s" totalImagesPullingTime="1.692947316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:17:47.510752035 +0000 UTC m=+276.578692510" watchObservedRunningTime="2026-07-16 00:17:47.511847763 +0000 UTC m=+276.579788239" Jul 16 00:17:48.456759 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:48.455923 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerStarted","Data":"c90a4747c1cd068ac34481c2cc910e4824412a55e84941259150f76d56a435d3"} Jul 16 00:17:49.463307 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:49.463265 2564 generic.go:356] "Generic (PLEG): container finished" podID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerID="c90a4747c1cd068ac34481c2cc910e4824412a55e84941259150f76d56a435d3" exitCode=0 Jul 16 00:17:49.463784 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:49.463318 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerDied","Data":"c90a4747c1cd068ac34481c2cc910e4824412a55e84941259150f76d56a435d3"} Jul 16 00:17:51.944230 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:51.943998 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" podUID="28871d19-a84d-41f4-8b5a-35e934555f75" containerName="registry" containerID="cri-o://96017031768b14e23984182da75065c0b53e6060aad4a9bd1da3ca5b457a9a2a" gracePeriod=30 Jul 16 00:17:52.481606 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:52.481564 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" event={"ID":"28871d19-a84d-41f4-8b5a-35e934555f75","Type":"ContainerDied","Data":"96017031768b14e23984182da75065c0b53e6060aad4a9bd1da3ca5b457a9a2a"} Jul 16 00:17:52.481606 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:52.481452 2564 generic.go:356] "Generic (PLEG): container finished" podID="28871d19-a84d-41f4-8b5a-35e934555f75" containerID="96017031768b14e23984182da75065c0b53e6060aad4a9bd1da3ca5b457a9a2a" exitCode=0 Jul 16 00:17:53.454895 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.454860 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:53.487125 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.487099 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" Jul 16 00:17:53.487285 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.487097 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87cff8cbd-fj9jz" event={"ID":"28871d19-a84d-41f4-8b5a-35e934555f75","Type":"ContainerDied","Data":"87573b0b83b4d55b495ad0598330d81a155d68a5a312626455a80ac9c8e914ad"} Jul 16 00:17:53.487285 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.487224 2564 scope.go:117] "RemoveContainer" containerID="96017031768b14e23984182da75065c0b53e6060aad4a9bd1da3ca5b457a9a2a" Jul 16 00:17:53.578187 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.578152 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/28871d19-a84d-41f4-8b5a-35e934555f75-ca-trust-extracted\") pod \"28871d19-a84d-41f4-8b5a-35e934555f75\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " Jul 16 00:17:53.578368 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.578207 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/28871d19-a84d-41f4-8b5a-35e934555f75-installation-pull-secrets\") pod \"28871d19-a84d-41f4-8b5a-35e934555f75\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " Jul 16 00:17:53.578368 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.578237 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92j4q\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-kube-api-access-92j4q\") pod \"28871d19-a84d-41f4-8b5a-35e934555f75\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " Jul 16 00:17:53.578368 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.578272 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/28871d19-a84d-41f4-8b5a-35e934555f75-trusted-ca\") pod \"28871d19-a84d-41f4-8b5a-35e934555f75\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " Jul 16 00:17:53.578368 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.578315 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-bound-sa-token\") pod \"28871d19-a84d-41f4-8b5a-35e934555f75\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " Jul 16 00:17:53.578368 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.578336 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls\") pod \"28871d19-a84d-41f4-8b5a-35e934555f75\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " Jul 16 00:17:53.578660 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.578401 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/28871d19-a84d-41f4-8b5a-35e934555f75-registry-certificates\") pod \"28871d19-a84d-41f4-8b5a-35e934555f75\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " Jul 16 00:17:53.578660 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.578430 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/28871d19-a84d-41f4-8b5a-35e934555f75-image-registry-private-configuration\") pod \"28871d19-a84d-41f4-8b5a-35e934555f75\" (UID: \"28871d19-a84d-41f4-8b5a-35e934555f75\") " Jul 16 00:17:53.578763 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.578712 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28871d19-a84d-41f4-8b5a-35e934555f75-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "28871d19-a84d-41f4-8b5a-35e934555f75" (UID: "28871d19-a84d-41f4-8b5a-35e934555f75"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:17:53.579040 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.579010 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28871d19-a84d-41f4-8b5a-35e934555f75-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "28871d19-a84d-41f4-8b5a-35e934555f75" (UID: "28871d19-a84d-41f4-8b5a-35e934555f75"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:17:53.581324 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.581294 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28871d19-a84d-41f4-8b5a-35e934555f75-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "28871d19-a84d-41f4-8b5a-35e934555f75" (UID: "28871d19-a84d-41f4-8b5a-35e934555f75"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:17:53.581916 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.581891 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28871d19-a84d-41f4-8b5a-35e934555f75-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "28871d19-a84d-41f4-8b5a-35e934555f75" (UID: "28871d19-a84d-41f4-8b5a-35e934555f75"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:17:53.582240 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.582200 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "28871d19-a84d-41f4-8b5a-35e934555f75" (UID: "28871d19-a84d-41f4-8b5a-35e934555f75"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:17:53.582982 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.582941 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "28871d19-a84d-41f4-8b5a-35e934555f75" (UID: "28871d19-a84d-41f4-8b5a-35e934555f75"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:17:53.583127 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.583100 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-kube-api-access-92j4q" (OuterVolumeSpecName: "kube-api-access-92j4q") pod "28871d19-a84d-41f4-8b5a-35e934555f75" (UID: "28871d19-a84d-41f4-8b5a-35e934555f75"). InnerVolumeSpecName "kube-api-access-92j4q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:17:53.590055 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.590024 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28871d19-a84d-41f4-8b5a-35e934555f75-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "28871d19-a84d-41f4-8b5a-35e934555f75" (UID: "28871d19-a84d-41f4-8b5a-35e934555f75"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:17:53.679883 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.679794 2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-bound-sa-token\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:17:53.679883 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.679830 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-registry-tls\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:17:53.679883 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.679843 2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/28871d19-a84d-41f4-8b5a-35e934555f75-registry-certificates\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:17:53.679883 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.679856 2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/28871d19-a84d-41f4-8b5a-35e934555f75-image-registry-private-configuration\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:17:53.679883 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.679870 2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/28871d19-a84d-41f4-8b5a-35e934555f75-ca-trust-extracted\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:17:53.679883 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.679882 2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/28871d19-a84d-41f4-8b5a-35e934555f75-installation-pull-secrets\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:17:53.680220 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.679894 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92j4q\" (UniqueName: \"kubernetes.io/projected/28871d19-a84d-41f4-8b5a-35e934555f75-kube-api-access-92j4q\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:17:53.680220 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.679906 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/28871d19-a84d-41f4-8b5a-35e934555f75-trusted-ca\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:17:53.812719 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.812681 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-87cff8cbd-fj9jz"] Jul 16 00:17:53.816806 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:53.816775 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-87cff8cbd-fj9jz"] Jul 16 00:17:54.495594 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:54.495528 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerStarted","Data":"b9c6a0eed0cdfc0ee29e9bdc1ea904cc65f11d29de9b22a8318f9304ccceedf5"} Jul 16 00:17:54.496030 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:54.495641 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerStarted","Data":"6b519f83bf31974e4044dcb12181b87e07d4b1693d8b132f752f10a6f5fe1bd3"} Jul 16 00:17:55.485101 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:55.485063 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28871d19-a84d-41f4-8b5a-35e934555f75" path="/var/lib/kubelet/pods/28871d19-a84d-41f4-8b5a-35e934555f75/volumes" Jul 16 00:17:55.799974 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:55.799936 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:17:55.802351 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:55.802324 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f5645e23-1659-4fa0-b4f8-7cc6e7f31d96-metrics-tls\") pod \"dns-default-k8wkl\" (UID: \"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96\") " pod="openshift-dns/dns-default-k8wkl" Jul 16 00:17:55.882828 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:55.882795 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-djqcn\"" Jul 16 00:17:55.890793 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:55.890766 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k8wkl" Jul 16 00:17:56.076372 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:56.076338 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k8wkl"] Jul 16 00:17:56.078916 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:17:56.078885 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5645e23_1659_4fa0_b4f8_7cc6e7f31d96.slice/crio-1d1d0392110f2ebca297b589ad1c3421328680557951ebc21942e684201a87a2 WatchSource:0}: Error finding container 1d1d0392110f2ebca297b589ad1c3421328680557951ebc21942e684201a87a2: Status 404 returned error can't find the container with id 1d1d0392110f2ebca297b589ad1c3421328680557951ebc21942e684201a87a2 Jul 16 00:17:56.506154 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:56.506068 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k8wkl" event={"ID":"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96","Type":"ContainerStarted","Data":"1d1d0392110f2ebca297b589ad1c3421328680557951ebc21942e684201a87a2"} Jul 16 00:17:57.515019 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:57.514979 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerStarted","Data":"c33e0fb2b5665470be8eb33abc2abcf1727757747525e1a68bcb7e54a635f7fb"} Jul 16 00:17:57.515404 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:57.515060 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerStarted","Data":"bf8fb50399c73a7c55975d201c906f203f7ad72e6f094b4ce1d55e3bcb9d17b6"} Jul 16 00:17:57.515404 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:57.515095 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerStarted","Data":"4acd56a1a5417c4540ee25563d741bbed426d6dac5d1716082fa3341eda91561"} Jul 16 00:17:58.525144 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:58.525113 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerStarted","Data":"dd14611512af6efcd53f17ca0faf6dd481d4831e500deeb4ce134a2a9dcb3acd"} Jul 16 00:17:58.553444 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:58.553376 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.357031506 podStartE2EDuration="15.553354526s" totalImagesPullingTime="9.174778649s" totalInitContainerRuntime="21.544371ms" isStatefulPod=false podCreationTimestamp="2026-07-16 00:17:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:17:58.551570357 +0000 UTC m=+287.619510834" watchObservedRunningTime="2026-07-16 00:17:58.553354526 +0000 UTC m=+287.621295003" Jul 16 00:17:59.530323 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:59.530289 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k8wkl" event={"ID":"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96","Type":"ContainerStarted","Data":"aa9c980ab3c0d368bf277887122f4a3b223aceb98f9ad66395a1570a4c108bfb"} Jul 16 00:17:59.530836 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:59.530347 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k8wkl" event={"ID":"f5645e23-1659-4fa0-b4f8-7cc6e7f31d96","Type":"ContainerStarted","Data":"bd028213c011db7114625713795115fe07dde82234698af5432125f4994e9317"} Jul 16 00:17:59.530836 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:59.530639 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-k8wkl" Jul 16 00:17:59.547153 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:17:59.547105 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-k8wkl" podStartSLOduration=251.719037331 podStartE2EDuration="4m14.547088774s" totalImagesPullingTime="2.828051443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:17:59.545946872 +0000 UTC m=+288.613887352" watchObservedRunningTime="2026-07-16 00:17:59.547088774 +0000 UTC m=+288.615029248" Jul 16 00:18:03.441171 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:03.441121 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:18:09.536343 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:09.536309 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-k8wkl" Jul 16 00:18:11.313226 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:11.313199 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:18:11.313873 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:11.313853 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:18:11.321693 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:11.321666 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:18:11.321952 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:11.321934 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:18:11.326042 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:11.326023 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 00:18:34.659729 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:34.659685 2564 generic.go:356] "Generic (PLEG): container finished" podID="6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5" containerID="0fc233acad64c4199b31c8fc0fba059a16adb9228049f78d0459331896a974de" exitCode=0 Jul 16 00:18:34.660153 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:34.659762 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" event={"ID":"6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5","Type":"ContainerDied","Data":"0fc233acad64c4199b31c8fc0fba059a16adb9228049f78d0459331896a974de"} Jul 16 00:18:34.660153 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:34.660123 2564 scope.go:117] "RemoveContainer" containerID="0fc233acad64c4199b31c8fc0fba059a16adb9228049f78d0459331896a974de" Jul 16 00:18:35.665242 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:35.665207 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bm2b9" event={"ID":"6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5","Type":"ContainerStarted","Data":"4dcc12fb25ecbb0d976f5e94e81a9652522109871f4c783bde6d8203bb95af4c"} Jul 16 00:18:43.440807 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:43.440767 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:18:43.461827 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:43.461800 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:18:43.718805 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:18:43.718723 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:01.751209 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:01.751178 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 00:19:01.751649 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:01.751450 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="prometheus" containerID="cri-o://6b519f83bf31974e4044dcb12181b87e07d4b1693d8b132f752f10a6f5fe1bd3" gracePeriod=600 Jul 16 00:19:01.751649 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:01.751501 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="kube-rbac-proxy" containerID="cri-o://c33e0fb2b5665470be8eb33abc2abcf1727757747525e1a68bcb7e54a635f7fb" gracePeriod=600 Jul 16 00:19:01.751649 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:01.751514 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="kube-rbac-proxy-web" containerID="cri-o://bf8fb50399c73a7c55975d201c906f203f7ad72e6f094b4ce1d55e3bcb9d17b6" gracePeriod=600 Jul 16 00:19:01.751649 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:01.751558 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="config-reloader" containerID="cri-o://b9c6a0eed0cdfc0ee29e9bdc1ea904cc65f11d29de9b22a8318f9304ccceedf5" gracePeriod=600 Jul 16 00:19:01.751649 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:01.751502 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="thanos-sidecar" containerID="cri-o://4acd56a1a5417c4540ee25563d741bbed426d6dac5d1716082fa3341eda91561" gracePeriod=600 Jul 16 00:19:01.751649 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:01.751605 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="kube-rbac-proxy-thanos" containerID="cri-o://dd14611512af6efcd53f17ca0faf6dd481d4831e500deeb4ce134a2a9dcb3acd" gracePeriod=600 Jul 16 00:19:02.774442 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:02.774348 2564 generic.go:356] "Generic (PLEG): container finished" podID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerID="dd14611512af6efcd53f17ca0faf6dd481d4831e500deeb4ce134a2a9dcb3acd" exitCode=0 Jul 16 00:19:02.774442 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:02.774373 2564 generic.go:356] "Generic (PLEG): container finished" podID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerID="c33e0fb2b5665470be8eb33abc2abcf1727757747525e1a68bcb7e54a635f7fb" exitCode=0 Jul 16 00:19:02.774442 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:02.774382 2564 generic.go:356] "Generic (PLEG): container finished" podID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerID="4acd56a1a5417c4540ee25563d741bbed426d6dac5d1716082fa3341eda91561" exitCode=0 Jul 16 00:19:02.774442 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:02.774389 2564 generic.go:356] "Generic (PLEG): container finished" podID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerID="b9c6a0eed0cdfc0ee29e9bdc1ea904cc65f11d29de9b22a8318f9304ccceedf5" exitCode=0 Jul 16 00:19:02.774442 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:02.774395 2564 generic.go:356] "Generic (PLEG): container finished" podID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerID="6b519f83bf31974e4044dcb12181b87e07d4b1693d8b132f752f10a6f5fe1bd3" exitCode=0 Jul 16 00:19:02.774442 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:02.774415 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerDied","Data":"dd14611512af6efcd53f17ca0faf6dd481d4831e500deeb4ce134a2a9dcb3acd"} Jul 16 00:19:02.774957 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:02.774479 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerDied","Data":"c33e0fb2b5665470be8eb33abc2abcf1727757747525e1a68bcb7e54a635f7fb"} Jul 16 00:19:02.774957 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:02.774507 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerDied","Data":"4acd56a1a5417c4540ee25563d741bbed426d6dac5d1716082fa3341eda91561"} Jul 16 00:19:02.774957 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:02.774547 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerDied","Data":"b9c6a0eed0cdfc0ee29e9bdc1ea904cc65f11d29de9b22a8318f9304ccceedf5"} Jul 16 00:19:02.774957 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:02.774578 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerDied","Data":"6b519f83bf31974e4044dcb12181b87e07d4b1693d8b132f752f10a6f5fe1bd3"} Jul 16 00:19:02.997718 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:02.997693 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.130387 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130291 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-0\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130387 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130340 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-db\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130387 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130357 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdqrd\" (UniqueName: \"kubernetes.io/projected/a48d61ed-d382-4b5f-9b59-bb34c9c20570-kube-api-access-cdqrd\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130387 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130377 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130740 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130394 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-metrics-client-certs\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130740 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130415 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130740 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130430 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-kube-rbac-proxy\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130740 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130445 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-2\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130740 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130493 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-metrics-client-ca\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130740 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130516 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-config\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130740 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130578 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-web-config\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130740 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130606 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-kubelet-serving-ca-bundle\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130740 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130631 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a48d61ed-d382-4b5f-9b59-bb34c9c20570-config-out\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130740 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130664 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-1\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130740 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130700 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-trusted-ca-bundle\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.130740 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130738 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a48d61ed-d382-4b5f-9b59-bb34c9c20570-tls-assets\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.131313 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130770 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-tls\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.131313 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130807 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-serving-certs-ca-bundle\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.131313 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130840 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-grpc-tls\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.131313 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.130870 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-thanos-prometheus-http-client-file\") pod \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\" (UID: \"a48d61ed-d382-4b5f-9b59-bb34c9c20570\") " Jul 16 00:19:03.131313 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.131125 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:19:03.131313 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.131194 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:19:03.131592 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.131388 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:19:03.131914 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.131807 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:19:03.132120 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.132088 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:19:03.132239 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.132208 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:19:03.132239 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.132223 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:19:03.132597 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.132459 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:19:03.132693 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.132618 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-db\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.132693 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.132639 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.132693 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.132654 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-metrics-client-ca\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.132693 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.132670 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.132693 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.132684 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.132936 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.132696 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.132936 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.132711 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.132936 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.132725 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a48d61ed-d382-4b5f-9b59-bb34c9c20570-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.134299 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.133826 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:19:03.134480 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.134439 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:19:03.134790 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.134762 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:19:03.134883 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.134783 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:19:03.135949 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.135921 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a48d61ed-d382-4b5f-9b59-bb34c9c20570-kube-api-access-cdqrd" (OuterVolumeSpecName: "kube-api-access-cdqrd") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "kube-api-access-cdqrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:19:03.136028 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.135968 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:19:03.136203 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.136167 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a48d61ed-d382-4b5f-9b59-bb34c9c20570-config-out" (OuterVolumeSpecName: "config-out") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:19:03.136203 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.136167 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-config" (OuterVolumeSpecName: "config") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:19:03.136348 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.136209 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a48d61ed-d382-4b5f-9b59-bb34c9c20570-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:19:03.136348 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.136307 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:19:03.136348 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.136326 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:19:03.145488 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.145465 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-web-config" (OuterVolumeSpecName: "web-config") pod "a48d61ed-d382-4b5f-9b59-bb34c9c20570" (UID: "a48d61ed-d382-4b5f-9b59-bb34c9c20570"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:19:03.233052 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.233014 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.233052 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.233044 2564 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-metrics-client-certs\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.233052 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.233055 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.233052 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.233064 2564 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-kube-rbac-proxy\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.233306 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.233073 2564 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-config\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.233306 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.233104 2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-web-config\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.233306 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.233112 2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a48d61ed-d382-4b5f-9b59-bb34c9c20570-config-out\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.233306 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.233120 2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a48d61ed-d382-4b5f-9b59-bb34c9c20570-tls-assets\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.233306 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.233128 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.233306 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.233135 2564 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-secret-grpc-tls\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.233306 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.233144 2564 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a48d61ed-d382-4b5f-9b59-bb34c9c20570-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.233306 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.233153 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdqrd\" (UniqueName: \"kubernetes.io/projected/a48d61ed-d382-4b5f-9b59-bb34c9c20570-kube-api-access-cdqrd\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:19:03.781393 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.781357 2564 generic.go:356] "Generic (PLEG): container finished" podID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerID="bf8fb50399c73a7c55975d201c906f203f7ad72e6f094b4ce1d55e3bcb9d17b6" exitCode=0 Jul 16 00:19:03.781816 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.781434 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerDied","Data":"bf8fb50399c73a7c55975d201c906f203f7ad72e6f094b4ce1d55e3bcb9d17b6"} Jul 16 00:19:03.781816 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.781449 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.781816 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.781504 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a48d61ed-d382-4b5f-9b59-bb34c9c20570","Type":"ContainerDied","Data":"de2bd1e1e58de4c6bb21a8bfa93c6857ed1f9afc1712671f8a5237556d3b89bb"} Jul 16 00:19:03.781816 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.781559 2564 scope.go:117] "RemoveContainer" containerID="dd14611512af6efcd53f17ca0faf6dd481d4831e500deeb4ce134a2a9dcb3acd" Jul 16 00:19:03.789360 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.789340 2564 scope.go:117] "RemoveContainer" containerID="c33e0fb2b5665470be8eb33abc2abcf1727757747525e1a68bcb7e54a635f7fb" Jul 16 00:19:03.796072 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.796055 2564 scope.go:117] "RemoveContainer" containerID="bf8fb50399c73a7c55975d201c906f203f7ad72e6f094b4ce1d55e3bcb9d17b6" Jul 16 00:19:03.805098 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.805075 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 00:19:03.807292 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.807269 2564 scope.go:117] "RemoveContainer" containerID="4acd56a1a5417c4540ee25563d741bbed426d6dac5d1716082fa3341eda91561" Jul 16 00:19:03.809589 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.809567 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 00:19:03.814161 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.814139 2564 scope.go:117] "RemoveContainer" containerID="b9c6a0eed0cdfc0ee29e9bdc1ea904cc65f11d29de9b22a8318f9304ccceedf5" Jul 16 00:19:03.820555 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.820515 2564 scope.go:117] "RemoveContainer" containerID="6b519f83bf31974e4044dcb12181b87e07d4b1693d8b132f752f10a6f5fe1bd3" Jul 16 00:19:03.827410 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.827381 2564 scope.go:117] "RemoveContainer" containerID="c90a4747c1cd068ac34481c2cc910e4824412a55e84941259150f76d56a435d3" Jul 16 00:19:03.834124 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.834105 2564 scope.go:117] "RemoveContainer" containerID="dd14611512af6efcd53f17ca0faf6dd481d4831e500deeb4ce134a2a9dcb3acd" Jul 16 00:19:03.834386 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:19:03.834367 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd14611512af6efcd53f17ca0faf6dd481d4831e500deeb4ce134a2a9dcb3acd\": container with ID starting with dd14611512af6efcd53f17ca0faf6dd481d4831e500deeb4ce134a2a9dcb3acd not found: ID does not exist" containerID="dd14611512af6efcd53f17ca0faf6dd481d4831e500deeb4ce134a2a9dcb3acd" Jul 16 00:19:03.834447 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.834396 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd14611512af6efcd53f17ca0faf6dd481d4831e500deeb4ce134a2a9dcb3acd"} err="failed to get container status \"dd14611512af6efcd53f17ca0faf6dd481d4831e500deeb4ce134a2a9dcb3acd\": rpc error: code = NotFound desc = could not find container \"dd14611512af6efcd53f17ca0faf6dd481d4831e500deeb4ce134a2a9dcb3acd\": container with ID starting with dd14611512af6efcd53f17ca0faf6dd481d4831e500deeb4ce134a2a9dcb3acd not found: ID does not exist" Jul 16 00:19:03.834447 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.834429 2564 scope.go:117] "RemoveContainer" containerID="c33e0fb2b5665470be8eb33abc2abcf1727757747525e1a68bcb7e54a635f7fb" Jul 16 00:19:03.834675 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:19:03.834660 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c33e0fb2b5665470be8eb33abc2abcf1727757747525e1a68bcb7e54a635f7fb\": container with ID starting with c33e0fb2b5665470be8eb33abc2abcf1727757747525e1a68bcb7e54a635f7fb not found: ID does not exist" containerID="c33e0fb2b5665470be8eb33abc2abcf1727757747525e1a68bcb7e54a635f7fb" Jul 16 00:19:03.834734 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.834678 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c33e0fb2b5665470be8eb33abc2abcf1727757747525e1a68bcb7e54a635f7fb"} err="failed to get container status \"c33e0fb2b5665470be8eb33abc2abcf1727757747525e1a68bcb7e54a635f7fb\": rpc error: code = NotFound desc = could not find container \"c33e0fb2b5665470be8eb33abc2abcf1727757747525e1a68bcb7e54a635f7fb\": container with ID starting with c33e0fb2b5665470be8eb33abc2abcf1727757747525e1a68bcb7e54a635f7fb not found: ID does not exist" Jul 16 00:19:03.834734 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.834690 2564 scope.go:117] "RemoveContainer" containerID="bf8fb50399c73a7c55975d201c906f203f7ad72e6f094b4ce1d55e3bcb9d17b6" Jul 16 00:19:03.834895 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:19:03.834876 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf8fb50399c73a7c55975d201c906f203f7ad72e6f094b4ce1d55e3bcb9d17b6\": container with ID starting with bf8fb50399c73a7c55975d201c906f203f7ad72e6f094b4ce1d55e3bcb9d17b6 not found: ID does not exist" containerID="bf8fb50399c73a7c55975d201c906f203f7ad72e6f094b4ce1d55e3bcb9d17b6" Jul 16 00:19:03.834935 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.834901 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8fb50399c73a7c55975d201c906f203f7ad72e6f094b4ce1d55e3bcb9d17b6"} err="failed to get container status \"bf8fb50399c73a7c55975d201c906f203f7ad72e6f094b4ce1d55e3bcb9d17b6\": rpc error: code = NotFound desc = could not find container \"bf8fb50399c73a7c55975d201c906f203f7ad72e6f094b4ce1d55e3bcb9d17b6\": container with ID starting with bf8fb50399c73a7c55975d201c906f203f7ad72e6f094b4ce1d55e3bcb9d17b6 not found: ID does not exist" Jul 16 00:19:03.834935 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.834918 2564 scope.go:117] "RemoveContainer" containerID="4acd56a1a5417c4540ee25563d741bbed426d6dac5d1716082fa3341eda91561" Jul 16 00:19:03.835127 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:19:03.835112 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4acd56a1a5417c4540ee25563d741bbed426d6dac5d1716082fa3341eda91561\": container with ID starting with 4acd56a1a5417c4540ee25563d741bbed426d6dac5d1716082fa3341eda91561 not found: ID does not exist" containerID="4acd56a1a5417c4540ee25563d741bbed426d6dac5d1716082fa3341eda91561" Jul 16 00:19:03.835165 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.835130 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4acd56a1a5417c4540ee25563d741bbed426d6dac5d1716082fa3341eda91561"} err="failed to get container status \"4acd56a1a5417c4540ee25563d741bbed426d6dac5d1716082fa3341eda91561\": rpc error: code = NotFound desc = could not find container \"4acd56a1a5417c4540ee25563d741bbed426d6dac5d1716082fa3341eda91561\": container with ID starting with 4acd56a1a5417c4540ee25563d741bbed426d6dac5d1716082fa3341eda91561 not found: ID does not exist" Jul 16 00:19:03.835165 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.835142 2564 scope.go:117] "RemoveContainer" containerID="b9c6a0eed0cdfc0ee29e9bdc1ea904cc65f11d29de9b22a8318f9304ccceedf5" Jul 16 00:19:03.835358 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:19:03.835341 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9c6a0eed0cdfc0ee29e9bdc1ea904cc65f11d29de9b22a8318f9304ccceedf5\": container with ID starting with b9c6a0eed0cdfc0ee29e9bdc1ea904cc65f11d29de9b22a8318f9304ccceedf5 not found: ID does not exist" containerID="b9c6a0eed0cdfc0ee29e9bdc1ea904cc65f11d29de9b22a8318f9304ccceedf5" Jul 16 00:19:03.835408 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.835360 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c6a0eed0cdfc0ee29e9bdc1ea904cc65f11d29de9b22a8318f9304ccceedf5"} err="failed to get container status \"b9c6a0eed0cdfc0ee29e9bdc1ea904cc65f11d29de9b22a8318f9304ccceedf5\": rpc error: code = NotFound desc = could not find container \"b9c6a0eed0cdfc0ee29e9bdc1ea904cc65f11d29de9b22a8318f9304ccceedf5\": container with ID starting with b9c6a0eed0cdfc0ee29e9bdc1ea904cc65f11d29de9b22a8318f9304ccceedf5 not found: ID does not exist" Jul 16 00:19:03.835408 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.835372 2564 scope.go:117] "RemoveContainer" containerID="6b519f83bf31974e4044dcb12181b87e07d4b1693d8b132f752f10a6f5fe1bd3" Jul 16 00:19:03.835590 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:19:03.835574 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b519f83bf31974e4044dcb12181b87e07d4b1693d8b132f752f10a6f5fe1bd3\": container with ID starting with 6b519f83bf31974e4044dcb12181b87e07d4b1693d8b132f752f10a6f5fe1bd3 not found: ID does not exist" containerID="6b519f83bf31974e4044dcb12181b87e07d4b1693d8b132f752f10a6f5fe1bd3" Jul 16 00:19:03.835637 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.835593 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b519f83bf31974e4044dcb12181b87e07d4b1693d8b132f752f10a6f5fe1bd3"} err="failed to get container status \"6b519f83bf31974e4044dcb12181b87e07d4b1693d8b132f752f10a6f5fe1bd3\": rpc error: code = NotFound desc = could not find container \"6b519f83bf31974e4044dcb12181b87e07d4b1693d8b132f752f10a6f5fe1bd3\": container with ID starting with 6b519f83bf31974e4044dcb12181b87e07d4b1693d8b132f752f10a6f5fe1bd3 not found: ID does not exist" Jul 16 00:19:03.835637 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.835605 2564 scope.go:117] "RemoveContainer" containerID="c90a4747c1cd068ac34481c2cc910e4824412a55e84941259150f76d56a435d3" Jul 16 00:19:03.835782 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:19:03.835768 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c90a4747c1cd068ac34481c2cc910e4824412a55e84941259150f76d56a435d3\": container with ID starting with c90a4747c1cd068ac34481c2cc910e4824412a55e84941259150f76d56a435d3 not found: ID does not exist" containerID="c90a4747c1cd068ac34481c2cc910e4824412a55e84941259150f76d56a435d3" Jul 16 00:19:03.835821 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.835786 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c90a4747c1cd068ac34481c2cc910e4824412a55e84941259150f76d56a435d3"} err="failed to get container status \"c90a4747c1cd068ac34481c2cc910e4824412a55e84941259150f76d56a435d3\": rpc error: code = NotFound desc = could not find container \"c90a4747c1cd068ac34481c2cc910e4824412a55e84941259150f76d56a435d3\": container with ID starting with c90a4747c1cd068ac34481c2cc910e4824412a55e84941259150f76d56a435d3 not found: ID does not exist" Jul 16 00:19:03.841908 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.841885 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 00:19:03.842139 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842126 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28871d19-a84d-41f4-8b5a-35e934555f75" containerName="registry" Jul 16 00:19:03.842139 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842140 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="28871d19-a84d-41f4-8b5a-35e934555f75" containerName="registry" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842153 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="kube-rbac-proxy" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842159 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="kube-rbac-proxy" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842172 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="thanos-sidecar" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842178 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="thanos-sidecar" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842183 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="kube-rbac-proxy-thanos" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842188 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="kube-rbac-proxy-thanos" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842196 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="config-reloader" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842201 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="config-reloader" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842207 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="kube-rbac-proxy-web" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842212 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="kube-rbac-proxy-web" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842218 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="init-config-reloader" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842223 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="init-config-reloader" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842229 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="prometheus" Jul 16 00:19:03.842254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842234 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="prometheus" Jul 16 00:19:03.842662 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842278 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="prometheus" Jul 16 00:19:03.842662 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842287 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="thanos-sidecar" Jul 16 00:19:03.842662 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842294 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="28871d19-a84d-41f4-8b5a-35e934555f75" containerName="registry" Jul 16 00:19:03.842662 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842299 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="kube-rbac-proxy" Jul 16 00:19:03.842662 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842305 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="config-reloader" Jul 16 00:19:03.842662 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842311 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="kube-rbac-proxy-thanos" Jul 16 00:19:03.842662 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.842316 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" containerName="kube-rbac-proxy-web" Jul 16 00:19:03.845784 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.845767 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.848936 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.848918 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 16 00:19:03.849016 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.848920 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 16 00:19:03.849375 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.849356 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 16 00:19:03.849736 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.849722 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 16 00:19:03.849794 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.849754 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 16 00:19:03.849842 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.849758 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 16 00:19:03.850370 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.850347 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 16 00:19:03.850370 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.850363 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ic81su8cd3nd\"" Jul 16 00:19:03.850501 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.850365 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 16 00:19:03.850773 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.850747 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 16 00:19:03.850841 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.850803 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 16 00:19:03.850935 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.850917 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 16 00:19:03.851050 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.851035 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 16 00:19:03.851329 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.851309 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 16 00:19:03.853780 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.853760 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hqx9j\"" Jul 16 00:19:03.859567 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.856326 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 16 00:19:03.860710 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.860693 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 16 00:19:03.871507 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.871487 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 00:19:03.938339 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938302 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dbb20ce4-f1f6-484e-b481-d270387cb6f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938339 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938338 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938526 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938362 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938526 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938421 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938526 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938454 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938526 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938473 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938526 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938490 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938526 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938509 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938734 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938564 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dbb20ce4-f1f6-484e-b481-d270387cb6f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938734 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938581 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938734 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938613 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938734 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938632 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938734 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938650 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938734 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938669 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938734 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938690 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb8x8\" (UniqueName: \"kubernetes.io/projected/dbb20ce4-f1f6-484e-b481-d270387cb6f9-kube-api-access-sb8x8\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938734 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938710 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938734 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938727 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938981 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-config\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938981 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938814 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:03.938981 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:03.938845 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039478 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039397 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039478 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039434 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039478 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039453 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039478 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039474 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039825 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039502 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb8x8\" (UniqueName: \"kubernetes.io/projected/dbb20ce4-f1f6-484e-b481-d270387cb6f9-kube-api-access-sb8x8\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039825 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039558 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039825 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039586 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039825 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039620 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-config\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039825 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039658 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039825 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039691 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039825 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039716 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dbb20ce4-f1f6-484e-b481-d270387cb6f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039825 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039744 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039825 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039768 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039825 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039794 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.039825 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039819 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.040345 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039843 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.040345 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039871 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.040345 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039910 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.040345 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039951 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dbb20ce4-f1f6-484e-b481-d270387cb6f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.040345 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.039977 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.040345 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.040199 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.040672 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.040381 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.040825 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.040804 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.041476 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.041450 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.042317 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.042197 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.042666 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.042525 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.042666 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.042648 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-config\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.042976 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.042951 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.043555 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.043512 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dbb20ce4-f1f6-484e-b481-d270387cb6f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.043670 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.043646 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.044059 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.044037 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.044142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.044038 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.044142 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.044083 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.044461 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.044436 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.044698 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.044674 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dbb20ce4-f1f6-484e-b481-d270387cb6f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.045029 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.045011 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.045281 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.045260 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.045804 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.045786 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dbb20ce4-f1f6-484e-b481-d270387cb6f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.046193 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.046178 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dbb20ce4-f1f6-484e-b481-d270387cb6f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.049130 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.049110 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb8x8\" (UniqueName: \"kubernetes.io/projected/dbb20ce4-f1f6-484e-b481-d270387cb6f9-kube-api-access-sb8x8\") pod \"prometheus-k8s-0\" (UID: \"dbb20ce4-f1f6-484e-b481-d270387cb6f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.155294 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.155256 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:19:04.308518 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.308298 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 00:19:04.311247 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:19:04.311219 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb20ce4_f1f6_484e_b481_d270387cb6f9.slice/crio-0a23ae30dad297435e2829f2efc823ebd54979715bef041b4aaacf5f5207aa09 WatchSource:0}: Error finding container 0a23ae30dad297435e2829f2efc823ebd54979715bef041b4aaacf5f5207aa09: Status 404 returned error can't find the container with id 0a23ae30dad297435e2829f2efc823ebd54979715bef041b4aaacf5f5207aa09 Jul 16 00:19:04.787090 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.787058 2564 generic.go:356] "Generic (PLEG): container finished" podID="dbb20ce4-f1f6-484e-b481-d270387cb6f9" containerID="a25a01256b3bacca2b0a9a0e16ae2c9e301137294125ceb85f93af97230ba686" exitCode=0 Jul 16 00:19:04.787480 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.787124 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbb20ce4-f1f6-484e-b481-d270387cb6f9","Type":"ContainerDied","Data":"a25a01256b3bacca2b0a9a0e16ae2c9e301137294125ceb85f93af97230ba686"} Jul 16 00:19:04.787480 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:04.787171 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbb20ce4-f1f6-484e-b481-d270387cb6f9","Type":"ContainerStarted","Data":"0a23ae30dad297435e2829f2efc823ebd54979715bef041b4aaacf5f5207aa09"} Jul 16 00:19:05.484024 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:05.483947 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a48d61ed-d382-4b5f-9b59-bb34c9c20570" path="/var/lib/kubelet/pods/a48d61ed-d382-4b5f-9b59-bb34c9c20570/volumes" Jul 16 00:19:05.794766 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:05.794735 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbb20ce4-f1f6-484e-b481-d270387cb6f9","Type":"ContainerStarted","Data":"274db845184a22df146e43daef0824124e549c4b73f2ccc9e54d63fa362685c9"} Jul 16 00:19:05.795155 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:05.794796 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbb20ce4-f1f6-484e-b481-d270387cb6f9","Type":"ContainerStarted","Data":"658b3249c05884d2278b265551e69957f85030f4ac01ae497f389bdfd60e21ae"} Jul 16 00:19:05.795155 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:05.794820 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbb20ce4-f1f6-484e-b481-d270387cb6f9","Type":"ContainerStarted","Data":"221edee3bdfeaffaff6a4c70128ea6052e3236f286a83fadaa7ddb7a0d1dfd38"} Jul 16 00:19:05.795155 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:05.794844 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbb20ce4-f1f6-484e-b481-d270387cb6f9","Type":"ContainerStarted","Data":"f21f93526f8985621042bd3da217f5de8fa9b011e22876d3d33d2c44eec0739a"} Jul 16 00:19:05.795155 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:05.794868 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbb20ce4-f1f6-484e-b481-d270387cb6f9","Type":"ContainerStarted","Data":"8052b4058e4b53c1b4a373fd4d9df6c15dacfeb4852e33e39871d8815e9c96b9"} Jul 16 00:19:05.795155 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:05.794889 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbb20ce4-f1f6-484e-b481-d270387cb6f9","Type":"ContainerStarted","Data":"f98793be16f56291b7b548a2cdee54c0be2598387ffd0922b39b3d98557b0478"} Jul 16 00:19:05.836811 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:05.836750 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.814142689 podStartE2EDuration="2.836726743s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.584054ms" isStatefulPod=false podCreationTimestamp="2026-07-16 00:19:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:19:05.835719365 +0000 UTC m=+354.903659841" watchObservedRunningTime="2026-07-16 00:19:05.836726743 +0000 UTC m=+354.904667219" Jul 16 00:19:09.156134 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:19:09.156101 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:20:04.156245 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:20:04.156202 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:20:04.171953 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:20:04.171926 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:20:05.026638 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:20:05.026610 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 00:22:04.333112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.333021 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4"] Jul 16 00:22:04.335614 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.335593 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4" Jul 16 00:22:04.338055 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.338035 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-vztzd\"" Jul 16 00:22:04.338161 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.338121 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 00:22:04.339190 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.339173 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 00:22:04.343961 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.343935 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4"] Jul 16 00:22:04.460334 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.460149 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/186d486a-064b-4297-92a4-eebf3c252208-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wwkf4\" (UID: \"186d486a-064b-4297-92a4-eebf3c252208\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4" Jul 16 00:22:04.460334 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.460218 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqk5h\" (UniqueName: \"kubernetes.io/projected/186d486a-064b-4297-92a4-eebf3c252208-kube-api-access-lqk5h\") pod \"openshift-lws-operator-fd8ccff4c-wwkf4\" (UID: \"186d486a-064b-4297-92a4-eebf3c252208\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4" Jul 16 00:22:04.561577 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.561518 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/186d486a-064b-4297-92a4-eebf3c252208-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wwkf4\" (UID: \"186d486a-064b-4297-92a4-eebf3c252208\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4" Jul 16 00:22:04.561756 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.561615 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqk5h\" (UniqueName: \"kubernetes.io/projected/186d486a-064b-4297-92a4-eebf3c252208-kube-api-access-lqk5h\") pod \"openshift-lws-operator-fd8ccff4c-wwkf4\" (UID: \"186d486a-064b-4297-92a4-eebf3c252208\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4" Jul 16 00:22:04.561943 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.561923 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/186d486a-064b-4297-92a4-eebf3c252208-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wwkf4\" (UID: \"186d486a-064b-4297-92a4-eebf3c252208\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4" Jul 16 00:22:04.571773 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.571743 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqk5h\" (UniqueName: \"kubernetes.io/projected/186d486a-064b-4297-92a4-eebf3c252208-kube-api-access-lqk5h\") pod \"openshift-lws-operator-fd8ccff4c-wwkf4\" (UID: \"186d486a-064b-4297-92a4-eebf3c252208\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4" Jul 16 00:22:04.645469 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.645381 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4" Jul 16 00:22:04.777746 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.777722 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4"] Jul 16 00:22:04.780402 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:22:04.780376 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod186d486a_064b_4297_92a4_eebf3c252208.slice/crio-51925ebd23c63ba43a55c530b19e1abf30b2cc2bb72a69a04f0488b9f42f547d WatchSource:0}: Error finding container 51925ebd23c63ba43a55c530b19e1abf30b2cc2bb72a69a04f0488b9f42f547d: Status 404 returned error can't find the container with id 51925ebd23c63ba43a55c530b19e1abf30b2cc2bb72a69a04f0488b9f42f547d Jul 16 00:22:04.782000 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:04.781977 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 00:22:05.461968 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:05.461933 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4" event={"ID":"186d486a-064b-4297-92a4-eebf3c252208","Type":"ContainerStarted","Data":"51925ebd23c63ba43a55c530b19e1abf30b2cc2bb72a69a04f0488b9f42f547d"} Jul 16 00:22:08.476745 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:08.476709 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4" event={"ID":"186d486a-064b-4297-92a4-eebf3c252208","Type":"ContainerStarted","Data":"15c08d44f5c77a3d7ddc56ff9276a0b83c5012334d74bd0259de30fa4e3fabcd"} Jul 16 00:22:08.496452 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:08.496400 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wwkf4" podStartSLOduration=1.777391482 podStartE2EDuration="4.496384385s" totalImagesPullingTime="2.718992903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:22:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:22:08.494850907 +0000 UTC m=+537.562791384" watchObservedRunningTime="2026-07-16 00:22:08.496384385 +0000 UTC m=+537.564324859" Jul 16 00:22:24.342004 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.341967 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl"] Jul 16 00:22:24.344301 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.344284 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.347254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.347188 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 00:22:24.347254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.347218 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pzpg8\"" Jul 16 00:22:24.347254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.347230 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 00:22:24.347254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.347256 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 00:22:24.347254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.347264 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 00:22:24.363343 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.363315 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl"] Jul 16 00:22:24.426930 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.426896 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0656155-9214-4f67-bbc0-623164c47b48-tmp\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.427112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.426977 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0656155-9214-4f67-bbc0-623164c47b48-webhook-cert\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.427112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.427018 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d0656155-9214-4f67-bbc0-623164c47b48-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.427112 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.427084 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc8d8\" (UniqueName: \"kubernetes.io/projected/d0656155-9214-4f67-bbc0-623164c47b48-kube-api-access-sc8d8\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.427264 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.427136 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d0656155-9214-4f67-bbc0-623164c47b48-manifests\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.527699 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.527663 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0656155-9214-4f67-bbc0-623164c47b48-webhook-cert\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.527699 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.527703 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d0656155-9214-4f67-bbc0-623164c47b48-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.527967 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.527721 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc8d8\" (UniqueName: \"kubernetes.io/projected/d0656155-9214-4f67-bbc0-623164c47b48-kube-api-access-sc8d8\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.527967 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.527744 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d0656155-9214-4f67-bbc0-623164c47b48-manifests\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.527967 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.527794 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0656155-9214-4f67-bbc0-623164c47b48-tmp\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.528229 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.528198 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d0656155-9214-4f67-bbc0-623164c47b48-manifests\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.530041 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.530012 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0656155-9214-4f67-bbc0-623164c47b48-tmp\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.530166 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.530147 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0656155-9214-4f67-bbc0-623164c47b48-webhook-cert\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.530232 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.530214 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d0656155-9214-4f67-bbc0-623164c47b48-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.543077 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.543045 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc8d8\" (UniqueName: \"kubernetes.io/projected/d0656155-9214-4f67-bbc0-623164c47b48-kube-api-access-sc8d8\") pod \"opendatahub-operator-controller-manager-dd579ccfb-55kdl\" (UID: \"d0656155-9214-4f67-bbc0-623164c47b48\") " pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.654495 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.654405 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:24.787103 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:24.787066 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl"] Jul 16 00:22:24.791361 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:22:24.791325 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0656155_9214_4f67_bbc0_623164c47b48.slice/crio-7a7cc9da5acc8a20ab43e7317a109d1cf2976152974b90bab70031c848950456 WatchSource:0}: Error finding container 7a7cc9da5acc8a20ab43e7317a109d1cf2976152974b90bab70031c848950456: Status 404 returned error can't find the container with id 7a7cc9da5acc8a20ab43e7317a109d1cf2976152974b90bab70031c848950456 Jul 16 00:22:25.548429 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:25.548384 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" event={"ID":"d0656155-9214-4f67-bbc0-623164c47b48","Type":"ContainerStarted","Data":"7a7cc9da5acc8a20ab43e7317a109d1cf2976152974b90bab70031c848950456"} Jul 16 00:22:28.565359 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:28.565309 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" event={"ID":"d0656155-9214-4f67-bbc0-623164c47b48","Type":"ContainerStarted","Data":"dc5b121c91178629ec3613b3319c47270ef44c757e18ab358e5f687a35376d76"} Jul 16 00:22:29.571269 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:29.571181 2564 generic.go:356] "Generic (PLEG): container finished" podID="d0656155-9214-4f67-bbc0-623164c47b48" containerID="dc5b121c91178629ec3613b3319c47270ef44c757e18ab358e5f687a35376d76" exitCode=0 Jul 16 00:22:29.571269 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:29.571248 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" event={"ID":"d0656155-9214-4f67-bbc0-623164c47b48","Type":"ContainerDied","Data":"dc5b121c91178629ec3613b3319c47270ef44c757e18ab358e5f687a35376d76"} Jul 16 00:22:30.578090 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:30.578051 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" event={"ID":"d0656155-9214-4f67-bbc0-623164c47b48","Type":"ContainerStarted","Data":"e63c1cfed90d321948db1609a3736283cbcae7c3fd771116aadce75b9faca21b"} Jul 16 00:22:30.578573 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:30.578135 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:30.602308 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:30.602256 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" podStartSLOduration=1.854318545 podStartE2EDuration="6.602238581s" totalImagesPullingTime="3.69741553s" totalInitContainerRuntime="1.050504506s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:22:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:22:30.600939486 +0000 UTC m=+559.668879961" watchObservedRunningTime="2026-07-16 00:22:30.602238581 +0000 UTC m=+559.670179056" Jul 16 00:22:41.586701 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:41.586673 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dd579ccfb-55kdl" Jul 16 00:22:49.959337 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:49.959297 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2"] Jul 16 00:22:49.962385 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:49.962367 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:49.964904 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:49.964883 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 00:22:49.965047 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:49.964919 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 00:22:49.965047 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:49.964974 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 00:22:49.965206 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:49.965190 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gc2l9\"" Jul 16 00:22:49.977713 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:49.977686 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2"] Jul 16 00:22:50.045017 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.044982 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.045017 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.045019 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.045263 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.045041 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.045263 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.045061 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqhgt\" (UniqueName: \"kubernetes.io/projected/10abc4f8-62a9-47ff-9058-d89bb8de1c94-kube-api-access-bqhgt\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.045263 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.045148 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.045263 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.045202 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.045405 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.045270 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.045405 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.045295 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.045405 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.045317 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146009 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.145965 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146009 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.146008 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.146031 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.146054 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqhgt\" (UniqueName: \"kubernetes.io/projected/10abc4f8-62a9-47ff-9058-d89bb8de1c94-kube-api-access-bqhgt\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.146077 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.146099 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.146139 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.146155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146255 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.146183 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146630 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.146435 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146630 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.146445 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146718 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.146642 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146845 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.146811 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.146961 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.146849 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.148416 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.148385 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.148685 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.148667 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.156883 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.156856 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10abc4f8-62a9-47ff-9058-d89bb8de1c94-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.157004 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.156895 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqhgt\" (UniqueName: \"kubernetes.io/projected/10abc4f8-62a9-47ff-9058-d89bb8de1c94-kube-api-access-bqhgt\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46kft2\" (UID: \"10abc4f8-62a9-47ff-9058-d89bb8de1c94\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.273701 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.273639 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:50.420899 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.420741 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2"] Jul 16 00:22:50.423600 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:22:50.423572 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10abc4f8_62a9_47ff_9058_d89bb8de1c94.slice/crio-03a06c575b4ec5a30a35120bde91db2480104a119b8c4dad1b9e98134c0d6719 WatchSource:0}: Error finding container 03a06c575b4ec5a30a35120bde91db2480104a119b8c4dad1b9e98134c0d6719: Status 404 returned error can't find the container with id 03a06c575b4ec5a30a35120bde91db2480104a119b8c4dad1b9e98134c0d6719 Jul 16 00:22:50.667823 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:50.667738 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" event={"ID":"10abc4f8-62a9-47ff-9058-d89bb8de1c94","Type":"ContainerStarted","Data":"03a06c575b4ec5a30a35120bde91db2480104a119b8c4dad1b9e98134c0d6719"} Jul 16 00:22:52.154452 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.154411 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98"] Jul 16 00:22:52.156890 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.156870 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.159385 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.159360 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 00:22:52.159385 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.159372 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 00:22:52.159587 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.159392 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-n2mpc\"" Jul 16 00:22:52.159587 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.159430 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 00:22:52.169265 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.169234 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98"] Jul 16 00:22:52.268329 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.268291 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f79ee54c-673f-4800-9fae-ece9824fb6eb-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.268498 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.268336 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f79ee54c-673f-4800-9fae-ece9824fb6eb-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.268498 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.268356 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f79ee54c-673f-4800-9fae-ece9824fb6eb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.268498 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.268439 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f79ee54c-673f-4800-9fae-ece9824fb6eb-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.268648 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.268525 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f79ee54c-673f-4800-9fae-ece9824fb6eb-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.268648 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.268574 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgrrw\" (UniqueName: \"kubernetes.io/projected/f79ee54c-673f-4800-9fae-ece9824fb6eb-kube-api-access-mgrrw\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.268648 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.268604 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f79ee54c-673f-4800-9fae-ece9824fb6eb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.369933 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.369895 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f79ee54c-673f-4800-9fae-ece9824fb6eb-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.370120 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.369948 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f79ee54c-673f-4800-9fae-ece9824fb6eb-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.370120 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.369969 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f79ee54c-673f-4800-9fae-ece9824fb6eb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.370242 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.370153 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f79ee54c-673f-4800-9fae-ece9824fb6eb-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.370304 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.370252 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f79ee54c-673f-4800-9fae-ece9824fb6eb-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.370304 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.370286 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgrrw\" (UniqueName: \"kubernetes.io/projected/f79ee54c-673f-4800-9fae-ece9824fb6eb-kube-api-access-mgrrw\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.370405 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.370333 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f79ee54c-673f-4800-9fae-ece9824fb6eb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.370884 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.370833 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f79ee54c-673f-4800-9fae-ece9824fb6eb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.372862 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.372836 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f79ee54c-673f-4800-9fae-ece9824fb6eb-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.372991 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.372932 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f79ee54c-673f-4800-9fae-ece9824fb6eb-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.373686 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.373665 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f79ee54c-673f-4800-9fae-ece9824fb6eb-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.373789 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.373763 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f79ee54c-673f-4800-9fae-ece9824fb6eb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.387968 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.387937 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgrrw\" (UniqueName: \"kubernetes.io/projected/f79ee54c-673f-4800-9fae-ece9824fb6eb-kube-api-access-mgrrw\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.389171 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.389146 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f79ee54c-673f-4800-9fae-ece9824fb6eb-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vkk98\" (UID: \"f79ee54c-673f-4800-9fae-ece9824fb6eb\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.466398 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.466279 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:52.663213 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.663182 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98"] Jul 16 00:22:52.664549 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:22:52.664508 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79ee54c_673f_4800_9fae_ece9824fb6eb.slice/crio-0d624798ea7f1af646fcb4b96d2311c0529f4cc796fdb9e94a48315ec03242bb WatchSource:0}: Error finding container 0d624798ea7f1af646fcb4b96d2311c0529f4cc796fdb9e94a48315ec03242bb: Status 404 returned error can't find the container with id 0d624798ea7f1af646fcb4b96d2311c0529f4cc796fdb9e94a48315ec03242bb Jul 16 00:22:52.681513 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:52.681460 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" event={"ID":"f79ee54c-673f-4800-9fae-ece9824fb6eb","Type":"ContainerStarted","Data":"0d624798ea7f1af646fcb4b96d2311c0529f4cc796fdb9e94a48315ec03242bb"} Jul 16 00:22:53.035261 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.035221 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd"] Jul 16 00:22:53.037775 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.037749 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.041883 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.041858 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9jngr\"" Jul 16 00:22:53.042812 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.042788 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 00:22:53.042812 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.042799 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 00:22:53.042970 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.042886 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 00:22:53.055645 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.055612 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd"] Jul 16 00:22:53.176955 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.176915 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9837e6d6-bc15-45c4-b3ba-b8f102cac92b-manager-config\") pod \"lws-controller-manager-58f9b8cc9c-g86sd\" (UID: \"9837e6d6-bc15-45c4-b3ba-b8f102cac92b\") " pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.177860 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.177823 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9837e6d6-bc15-45c4-b3ba-b8f102cac92b-cert\") pod \"lws-controller-manager-58f9b8cc9c-g86sd\" (UID: \"9837e6d6-bc15-45c4-b3ba-b8f102cac92b\") " pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.178016 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.177897 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkqh7\" (UniqueName: \"kubernetes.io/projected/9837e6d6-bc15-45c4-b3ba-b8f102cac92b-kube-api-access-wkqh7\") pod \"lws-controller-manager-58f9b8cc9c-g86sd\" (UID: \"9837e6d6-bc15-45c4-b3ba-b8f102cac92b\") " pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.178016 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.177954 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9837e6d6-bc15-45c4-b3ba-b8f102cac92b-metrics-cert\") pod \"lws-controller-manager-58f9b8cc9c-g86sd\" (UID: \"9837e6d6-bc15-45c4-b3ba-b8f102cac92b\") " pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.278652 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.278523 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9837e6d6-bc15-45c4-b3ba-b8f102cac92b-cert\") pod \"lws-controller-manager-58f9b8cc9c-g86sd\" (UID: \"9837e6d6-bc15-45c4-b3ba-b8f102cac92b\") " pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.278652 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.278637 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkqh7\" (UniqueName: \"kubernetes.io/projected/9837e6d6-bc15-45c4-b3ba-b8f102cac92b-kube-api-access-wkqh7\") pod \"lws-controller-manager-58f9b8cc9c-g86sd\" (UID: \"9837e6d6-bc15-45c4-b3ba-b8f102cac92b\") " pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.278968 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.278703 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9837e6d6-bc15-45c4-b3ba-b8f102cac92b-metrics-cert\") pod \"lws-controller-manager-58f9b8cc9c-g86sd\" (UID: \"9837e6d6-bc15-45c4-b3ba-b8f102cac92b\") " pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.278968 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.278752 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9837e6d6-bc15-45c4-b3ba-b8f102cac92b-manager-config\") pod \"lws-controller-manager-58f9b8cc9c-g86sd\" (UID: \"9837e6d6-bc15-45c4-b3ba-b8f102cac92b\") " pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.279562 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.279509 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9837e6d6-bc15-45c4-b3ba-b8f102cac92b-manager-config\") pod \"lws-controller-manager-58f9b8cc9c-g86sd\" (UID: \"9837e6d6-bc15-45c4-b3ba-b8f102cac92b\") " pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.282239 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.282179 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9837e6d6-bc15-45c4-b3ba-b8f102cac92b-cert\") pod \"lws-controller-manager-58f9b8cc9c-g86sd\" (UID: \"9837e6d6-bc15-45c4-b3ba-b8f102cac92b\") " pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.282378 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.282326 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9837e6d6-bc15-45c4-b3ba-b8f102cac92b-metrics-cert\") pod \"lws-controller-manager-58f9b8cc9c-g86sd\" (UID: \"9837e6d6-bc15-45c4-b3ba-b8f102cac92b\") " pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.290283 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.290213 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkqh7\" (UniqueName: \"kubernetes.io/projected/9837e6d6-bc15-45c4-b3ba-b8f102cac92b-kube-api-access-wkqh7\") pod \"lws-controller-manager-58f9b8cc9c-g86sd\" (UID: \"9837e6d6-bc15-45c4-b3ba-b8f102cac92b\") " pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.352193 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.352157 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:53.573979 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.573904 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd"] Jul 16 00:22:53.575327 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:22:53.575293 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9837e6d6_bc15_45c4_b3ba_b8f102cac92b.slice/crio-ba5b03e29a407160aa41cac72dbfd3223d41b59258492ee5d982e5785ac0d354 WatchSource:0}: Error finding container ba5b03e29a407160aa41cac72dbfd3223d41b59258492ee5d982e5785ac0d354: Status 404 returned error can't find the container with id ba5b03e29a407160aa41cac72dbfd3223d41b59258492ee5d982e5785ac0d354 Jul 16 00:22:53.688304 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:53.688271 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" event={"ID":"9837e6d6-bc15-45c4-b3ba-b8f102cac92b","Type":"ContainerStarted","Data":"ba5b03e29a407160aa41cac72dbfd3223d41b59258492ee5d982e5785ac0d354"} Jul 16 00:22:55.788955 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:55.788889 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:22:55.789265 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:55.788993 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:22:55.789265 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:55.789048 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:22:55.790575 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:55.790527 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:22:55.790663 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:55.790607 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:22:56.708473 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:56.708437 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" event={"ID":"f79ee54c-673f-4800-9fae-ece9824fb6eb","Type":"ContainerStarted","Data":"07c37f4bc28f106df9242f392c13e5923cb94e678ec66c77a45defdda70438ef"} Jul 16 00:22:56.708806 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:56.708785 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:56.710729 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:56.710687 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" event={"ID":"10abc4f8-62a9-47ff-9058-d89bb8de1c94","Type":"ContainerStarted","Data":"9aae2237027e6cea08a0f65601eff8238b7defc50bef447d16124cd8f9eaabc3"} Jul 16 00:22:56.710926 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:56.710903 2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-vkk98 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 00:22:56.711012 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:56.710957 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" podUID="f79ee54c-673f-4800-9fae-ece9824fb6eb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 00:22:56.736909 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:56.736855 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" podStartSLOduration=1.613701674 podStartE2EDuration="4.73682664s" totalImagesPullingTime="3.123124966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:22:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:22:56.736032319 +0000 UTC m=+585.803972799" watchObservedRunningTime="2026-07-16 00:22:56.73682664 +0000 UTC m=+585.804767114" Jul 16 00:22:56.765288 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:56.765232 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" podStartSLOduration=2.402275163 podStartE2EDuration="7.76521272s" totalImagesPullingTime="5.362937557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:22:56.763849286 +0000 UTC m=+585.831789752" watchObservedRunningTime="2026-07-16 00:22:56.76521272 +0000 UTC m=+585.833153253" Jul 16 00:22:57.274473 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:57.274428 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:57.279732 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:57.279704 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:57.716518 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:57.716416 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" event={"ID":"9837e6d6-bc15-45c4-b3ba-b8f102cac92b","Type":"ContainerStarted","Data":"d85c972529facf057842e38eddfba8be963a3ab74f6ea48842af8e8705e40c9a"} Jul 16 00:22:57.716680 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:57.716635 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:22:57.716680 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:57.716675 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:57.717443 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:57.717421 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vkk98" Jul 16 00:22:57.717747 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:57.717731 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46kft2" Jul 16 00:22:57.734868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:22:57.734818 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" podStartSLOduration=1.544463105 podStartE2EDuration="4.734801136s" totalImagesPullingTime="3.190338031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:22:57.73456711 +0000 UTC m=+586.802507582" watchObservedRunningTime="2026-07-16 00:22:57.734801136 +0000 UTC m=+586.802741612" Jul 16 00:23:08.723126 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:23:08.723096 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58f9b8cc9c-g86sd" Jul 16 00:23:11.346379 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:23:11.344943 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:23:11.347650 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:23:11.347615 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:23:11.355443 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:23:11.355421 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:23:11.356519 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:23:11.356501 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:25:32.274983 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.274944 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:25:32.277508 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.277491 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-vtmws" Jul 16 00:25:32.293810 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.293769 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 00:25:32.294924 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.294907 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 00:25:32.295011 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.294910 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 16 00:25:32.295595 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.295581 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-95lhl\"" Jul 16 00:25:32.308549 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.308509 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:25:32.378206 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.378168 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:25:32.401366 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.401335 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807-config-file\") pod \"limitador-limitador-54d6d988d5-vtmws\" (UID: \"4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vtmws" Jul 16 00:25:32.401366 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.401368 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scrkg\" (UniqueName: \"kubernetes.io/projected/4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807-kube-api-access-scrkg\") pod \"limitador-limitador-54d6d988d5-vtmws\" (UID: \"4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vtmws" Jul 16 00:25:32.502357 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.502321 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807-config-file\") pod \"limitador-limitador-54d6d988d5-vtmws\" (UID: \"4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vtmws" Jul 16 00:25:32.502357 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.502358 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scrkg\" (UniqueName: \"kubernetes.io/projected/4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807-kube-api-access-scrkg\") pod \"limitador-limitador-54d6d988d5-vtmws\" (UID: \"4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vtmws" Jul 16 00:25:32.502970 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.502950 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807-config-file\") pod \"limitador-limitador-54d6d988d5-vtmws\" (UID: \"4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vtmws" Jul 16 00:25:32.512905 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.512884 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scrkg\" (UniqueName: \"kubernetes.io/projected/4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807-kube-api-access-scrkg\") pod \"limitador-limitador-54d6d988d5-vtmws\" (UID: \"4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vtmws" Jul 16 00:25:32.586917 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.586830 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-vtmws" Jul 16 00:25:32.733998 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:32.733971 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:25:32.735795 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:25:32.735774 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d7292d3_ce4f_4dcd_bf7d_c171cd9ad807.slice/crio-b134c0353ccb6fc62886a3f58452d0d0f9a24a8a02411ac84c7c72d935d8770b WatchSource:0}: Error finding container b134c0353ccb6fc62886a3f58452d0d0f9a24a8a02411ac84c7c72d935d8770b: Status 404 returned error can't find the container with id b134c0353ccb6fc62886a3f58452d0d0f9a24a8a02411ac84c7c72d935d8770b Jul 16 00:25:33.396823 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:33.396780 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-vtmws" event={"ID":"4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807","Type":"ContainerStarted","Data":"b134c0353ccb6fc62886a3f58452d0d0f9a24a8a02411ac84c7c72d935d8770b"} Jul 16 00:25:36.411405 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:36.411366 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-vtmws" event={"ID":"4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807","Type":"ContainerStarted","Data":"6b1e71e49ae409eeb8c0775aaaa3be1aeec9bb400463a29af2f79eff2df38d37"} Jul 16 00:25:36.411797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:36.411474 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-vtmws" Jul 16 00:25:36.438135 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:36.438085 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-vtmws" podStartSLOduration=1.737565424 podStartE2EDuration="4.438069082s" totalImagesPullingTime="2.700503658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:25:36.436829128 +0000 UTC m=+745.504769607" watchObservedRunningTime="2026-07-16 00:25:36.438069082 +0000 UTC m=+745.506009557" Jul 16 00:25:47.416865 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:25:47.416832 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-vtmws" Jul 16 00:26:49.653704 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:26:49.653628 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:27:07.074224 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:07.074186 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:27:19.653458 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.653427 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k"] Jul 16 00:27:19.656968 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.656950 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.662525 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.662497 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 00:27:19.662986 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.662964 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p88nw\"" Jul 16 00:27:19.663110 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.663010 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 16 00:27:19.671521 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.671494 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 00:27:19.722254 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.722219 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k"] Jul 16 00:27:19.737894 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.737860 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de2e3873-bae7-47ae-9c82-8b7e33c39972-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.737894 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.737894 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de2e3873-bae7-47ae-9c82-8b7e33c39972-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.738083 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.737997 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de2e3873-bae7-47ae-9c82-8b7e33c39972-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.738083 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.738036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de2e3873-bae7-47ae-9c82-8b7e33c39972-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.738083 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.738060 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk8hs\" (UniqueName: \"kubernetes.io/projected/de2e3873-bae7-47ae-9c82-8b7e33c39972-kube-api-access-qk8hs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.738186 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.738088 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de2e3873-bae7-47ae-9c82-8b7e33c39972-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.839524 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.839479 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de2e3873-bae7-47ae-9c82-8b7e33c39972-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.839720 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.839562 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de2e3873-bae7-47ae-9c82-8b7e33c39972-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.839720 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.839591 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk8hs\" (UniqueName: \"kubernetes.io/projected/de2e3873-bae7-47ae-9c82-8b7e33c39972-kube-api-access-qk8hs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.839720 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.839631 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de2e3873-bae7-47ae-9c82-8b7e33c39972-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.839720 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.839697 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de2e3873-bae7-47ae-9c82-8b7e33c39972-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.839720 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.839719 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de2e3873-bae7-47ae-9c82-8b7e33c39972-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.840080 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.840055 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de2e3873-bae7-47ae-9c82-8b7e33c39972-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.840080 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.840076 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de2e3873-bae7-47ae-9c82-8b7e33c39972-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.840189 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.840115 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de2e3873-bae7-47ae-9c82-8b7e33c39972-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.841867 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.841844 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de2e3873-bae7-47ae-9c82-8b7e33c39972-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.842048 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.842034 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de2e3873-bae7-47ae-9c82-8b7e33c39972-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.849400 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.849375 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk8hs\" (UniqueName: \"kubernetes.io/projected/de2e3873-bae7-47ae-9c82-8b7e33c39972-kube-api-access-qk8hs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qbh4k\" (UID: \"de2e3873-bae7-47ae-9c82-8b7e33c39972\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:19.966693 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:19.966588 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:20.108786 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:20.108749 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k"] Jul 16 00:27:20.112238 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:27:20.112209 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde2e3873_bae7_47ae_9c82_8b7e33c39972.slice/crio-1bac3b037fe91cd5e8cd865e7f8510c43077d62d81f79b11ab09b191e895b41b WatchSource:0}: Error finding container 1bac3b037fe91cd5e8cd865e7f8510c43077d62d81f79b11ab09b191e895b41b: Status 404 returned error can't find the container with id 1bac3b037fe91cd5e8cd865e7f8510c43077d62d81f79b11ab09b191e895b41b Jul 16 00:27:20.113497 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:20.113478 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 00:27:20.545932 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:20.545898 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:27:20.875008 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:20.874915 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" event={"ID":"de2e3873-bae7-47ae-9c82-8b7e33c39972","Type":"ContainerStarted","Data":"1bac3b037fe91cd5e8cd865e7f8510c43077d62d81f79b11ab09b191e895b41b"} Jul 16 00:27:21.880781 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:21.880745 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" event={"ID":"de2e3873-bae7-47ae-9c82-8b7e33c39972","Type":"ContainerStarted","Data":"c2f24b684cf5c28582573c8b0e6cc02cc733117554714d3ba6b9ec5e44c5c5d1"} Jul 16 00:27:29.967037 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:29.967000 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:29.967553 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:29.967075 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:29.980276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:29.980247 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:30.013135 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:30.013085 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" podStartSLOduration=9.939313252 podStartE2EDuration="11.013069703s" totalImagesPullingTime="1.073756451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:27:21.903389307 +0000 UTC m=+850.971329784" watchObservedRunningTime="2026-07-16 00:27:30.013069703 +0000 UTC m=+859.081010178" Jul 16 00:27:30.932400 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:30.932371 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qbh4k" Jul 16 00:27:32.739555 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:32.739504 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:27:51.251058 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:51.251025 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:27:53.219122 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.219076 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5"] Jul 16 00:27:53.222896 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.222873 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.225389 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.225367 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 16 00:27:53.231411 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.231385 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5"] Jul 16 00:27:53.340891 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.340844 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edf2d7a0-43b8-4501-a145-66c90136c0fd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.340891 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.340893 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/edf2d7a0-43b8-4501-a145-66c90136c0fd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.341101 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.340921 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/edf2d7a0-43b8-4501-a145-66c90136c0fd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.341101 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.340994 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/edf2d7a0-43b8-4501-a145-66c90136c0fd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.341101 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.341051 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/edf2d7a0-43b8-4501-a145-66c90136c0fd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.341101 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.341087 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnsmc\" (UniqueName: \"kubernetes.io/projected/edf2d7a0-43b8-4501-a145-66c90136c0fd-kube-api-access-wnsmc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.442465 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.442410 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edf2d7a0-43b8-4501-a145-66c90136c0fd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.442682 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.442476 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/edf2d7a0-43b8-4501-a145-66c90136c0fd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.442682 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.442513 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/edf2d7a0-43b8-4501-a145-66c90136c0fd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.442682 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.442573 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/edf2d7a0-43b8-4501-a145-66c90136c0fd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.442682 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.442621 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/edf2d7a0-43b8-4501-a145-66c90136c0fd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.442682 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.442668 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnsmc\" (UniqueName: \"kubernetes.io/projected/edf2d7a0-43b8-4501-a145-66c90136c0fd-kube-api-access-wnsmc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.442943 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.442864 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edf2d7a0-43b8-4501-a145-66c90136c0fd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.443075 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.443055 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/edf2d7a0-43b8-4501-a145-66c90136c0fd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.443141 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.443062 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/edf2d7a0-43b8-4501-a145-66c90136c0fd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.444849 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.444828 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/edf2d7a0-43b8-4501-a145-66c90136c0fd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.445094 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.445075 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/edf2d7a0-43b8-4501-a145-66c90136c0fd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.451188 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.451163 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnsmc\" (UniqueName: \"kubernetes.io/projected/edf2d7a0-43b8-4501-a145-66c90136c0fd-kube-api-access-wnsmc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-skkz5\" (UID: \"edf2d7a0-43b8-4501-a145-66c90136c0fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.535110 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.535065 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:27:53.684522 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:53.684486 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5"] Jul 16 00:27:53.689396 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:27:53.689369 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedf2d7a0_43b8_4501_a145_66c90136c0fd.slice/crio-e864eb6c998419ea223c8146cf62a5a7bafa657a5513643297c8a5e7cc36cb0d WatchSource:0}: Error finding container e864eb6c998419ea223c8146cf62a5a7bafa657a5513643297c8a5e7cc36cb0d: Status 404 returned error can't find the container with id e864eb6c998419ea223c8146cf62a5a7bafa657a5513643297c8a5e7cc36cb0d Jul 16 00:27:54.026560 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:54.026492 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" event={"ID":"edf2d7a0-43b8-4501-a145-66c90136c0fd","Type":"ContainerStarted","Data":"f5cfe7898366aa5d3f3af5f89787a259557f02e074f8b839a661351f2a21b873"} Jul 16 00:27:54.026730 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:54.026575 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" event={"ID":"edf2d7a0-43b8-4501-a145-66c90136c0fd","Type":"ContainerStarted","Data":"e864eb6c998419ea223c8146cf62a5a7bafa657a5513643297c8a5e7cc36cb0d"} Jul 16 00:27:54.057929 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:27:54.057874 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" podStartSLOduration=0.889558859 podStartE2EDuration="1.057858891s" totalImagesPullingTime="168.300032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:27:54.056923594 +0000 UTC m=+883.124864068" watchObservedRunningTime="2026-07-16 00:27:54.057858891 +0000 UTC m=+883.125799365" Jul 16 00:28:00.446522 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:28:00.446484 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:28:03.536095 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:28:03.536010 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:28:03.536450 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:28:03.536340 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:28:03.548389 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:28:03.548366 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:28:04.081172 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:28:04.081140 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-skkz5" Jul 16 00:28:08.252271 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:28:08.252232 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:28:11.375400 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:28:11.375374 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:28:11.378126 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:28:11.378100 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:28:11.386162 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:28:11.386138 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:28:11.394843 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:28:11.394812 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:29:30.652456 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:29:30.652410 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:29:40.637627 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:29:40.637526 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:29:52.043464 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:29:52.043428 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:30:00.959471 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:30:00.959428 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:30:11.182661 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:30:11.182625 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:30:38.556157 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:30:38.556124 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:30:51.342439 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:30:51.342406 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:31:28.795631 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:31:28.795593 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:31:39.852639 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:31:39.852600 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:31:48.458453 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:31:48.458412 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:31:58.239995 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:31:58.239957 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:32:10.109570 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:32:10.109516 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:32:19.168475 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:32:19.168442 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:32:30.760788 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:32:30.760751 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:32:41.144049 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:32:41.143965 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:33:06.303093 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:33:06.303062 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:33:11.412858 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:33:11.412825 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:33:11.416623 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:33:11.416589 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:33:11.421807 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:33:11.421785 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:33:11.424788 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:33:11.424772 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:33:17.669667 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:33:17.669627 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:33:41.753505 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:33:41.753471 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:33:55.348559 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:33:55.348513 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:34:09.147630 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:34:09.147592 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:34:26.145904 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:34:26.145869 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:34:35.346757 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:34:35.346716 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:34:45.147863 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:34:45.147829 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:36:31.660189 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:36:31.660149 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:36:41.563382 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:36:41.563348 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:36:54.865809 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:36:54.865769 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:37:05.466898 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:37:05.466817 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:37:16.241986 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:37:16.241946 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:37:28.030838 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:37:28.030803 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:37:36.765439 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:37:36.765401 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:37:48.545797 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:37:48.545762 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:37:57.543595 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:37:57.543562 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:38:08.438053 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:38:08.438018 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:38:11.442119 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:38:11.442090 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:38:11.445727 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:38:11.445703 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:38:11.453340 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:38:11.453312 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:38:11.456210 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:38:11.456187 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:38:21.053062 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:38:21.053026 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:38:42.046197 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:38:42.046114 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:38:51.546383 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:38:51.546347 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:39:01.151434 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:39:01.151390 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:39:12.157686 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:39:12.157637 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:39:29.243982 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:39:29.243943 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:39:39.154391 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:39:39.154340 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:39:51.540184 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:39:51.540153 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:40:02.086341 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:40:02.086300 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:40:11.033356 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:40:11.033320 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:40:20.949317 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:40:20.949280 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:40:33.936622 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:40:33.936578 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:40:43.541683 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:40:43.541647 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:40:53.436935 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:40:53.436895 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:41:04.137771 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:41:04.137736 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:41:18.136374 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:41:18.136338 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:41:27.836473 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:41:27.836440 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:41:40.249782 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:41:40.249701 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:41:49.649135 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:41:49.649095 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:41:58.836490 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:41:58.836456 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:42:05.775097 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:42:05.775056 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:42:19.443910 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:42:19.443863 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:42:28.296678 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:42:28.296643 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:42:38.646472 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:42:38.646437 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:42:57.998404 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:42:57.998366 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:43:08.559152 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:43:08.559113 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:43:11.480262 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:43:11.480227 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:43:11.484999 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:43:11.484977 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:43:11.488444 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:43:11.488419 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:43:11.493378 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:43:11.493359 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:43:18.282630 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:43:18.282593 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:44:43.069700 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:44:43.069614 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:44:52.751756 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:44:52.751723 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:45:00.149361 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:00.149321 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-0aafd8c9-29736045-jvfsj"] Jul 16 00:45:00.153003 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:00.152975 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-0aafd8c9-29736045-jvfsj" Jul 16 00:45:00.156716 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:00.156688 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qks69\"" Jul 16 00:45:00.163172 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:00.163146 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-0aafd8c9-29736045-jvfsj" Jul 16 00:45:00.177205 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:00.177131 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-0aafd8c9-29736045-jvfsj"] Jul 16 00:45:00.508119 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:00.508097 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-0aafd8c9-29736045-jvfsj"] Jul 16 00:45:00.510372 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:45:00.510347 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde0017b7_e081_426b_b95a_1d1485294665.slice/crio-a8db4ff2b0fb371b41a06bccd8b7d0203a7daae7f58fa7988d246fdbcbd9fb9d WatchSource:0}: Error finding container a8db4ff2b0fb371b41a06bccd8b7d0203a7daae7f58fa7988d246fdbcbd9fb9d: Status 404 returned error can't find the container with id a8db4ff2b0fb371b41a06bccd8b7d0203a7daae7f58fa7988d246fdbcbd9fb9d Jul 16 00:45:00.512699 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:00.512684 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 00:45:00.675492 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:00.675453 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-0aafd8c9-29736045-jvfsj" event={"ID":"de0017b7-e081-426b-b95a-1d1485294665","Type":"ContainerStarted","Data":"a8db4ff2b0fb371b41a06bccd8b7d0203a7daae7f58fa7988d246fdbcbd9fb9d"} Jul 16 00:45:02.451039 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:02.451001 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:45:03.694221 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:03.694186 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-0aafd8c9-29736045-jvfsj" event={"ID":"de0017b7-e081-426b-b95a-1d1485294665","Type":"ContainerStarted","Data":"71bedf469ee316bcbc6c16d321ab2f561ede97bdb5516c3a3d51b4a7be33eb03"} Jul 16 00:45:03.711155 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:03.711101 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-0aafd8c9-29736045-jvfsj" podStartSLOduration=1.374896328 podStartE2EDuration="3.711081656s" totalImagesPullingTime="2.336185328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:45:03.71017399 +0000 UTC m=+1912.778114465" watchObservedRunningTime="2026-07-16 00:45:03.711081656 +0000 UTC m=+1912.779022134" Jul 16 00:45:04.700460 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:04.700423 2564 generic.go:356] "Generic (PLEG): container finished" podID="de0017b7-e081-426b-b95a-1d1485294665" containerID="71bedf469ee316bcbc6c16d321ab2f561ede97bdb5516c3a3d51b4a7be33eb03" exitCode=0 Jul 16 00:45:04.700926 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:04.700507 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-0aafd8c9-29736045-jvfsj" event={"ID":"de0017b7-e081-426b-b95a-1d1485294665","Type":"ContainerDied","Data":"71bedf469ee316bcbc6c16d321ab2f561ede97bdb5516c3a3d51b4a7be33eb03"} Jul 16 00:45:05.839329 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:05.839298 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-0aafd8c9-29736045-jvfsj" Jul 16 00:45:06.711288 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:06.711261 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-0aafd8c9-29736045-jvfsj" Jul 16 00:45:06.711468 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:06.711310 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-0aafd8c9-29736045-jvfsj" event={"ID":"de0017b7-e081-426b-b95a-1d1485294665","Type":"ContainerDied","Data":"a8db4ff2b0fb371b41a06bccd8b7d0203a7daae7f58fa7988d246fdbcbd9fb9d"} Jul 16 00:45:06.711468 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:06.711356 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8db4ff2b0fb371b41a06bccd8b7d0203a7daae7f58fa7988d246fdbcbd9fb9d" Jul 16 00:45:22.550089 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:22.550051 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:45:43.956551 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:43.956501 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:45:55.985916 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:45:55.985880 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:46:43.593942 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.593909 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg"] Jul 16 00:46:43.594397 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.594242 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de0017b7-e081-426b-b95a-1d1485294665" containerName="cleanup" Jul 16 00:46:43.594397 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.594252 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="de0017b7-e081-426b-b95a-1d1485294665" containerName="cleanup" Jul 16 00:46:43.594397 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.594308 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="de0017b7-e081-426b-b95a-1d1485294665" containerName="cleanup" Jul 16 00:46:43.597276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.597259 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.600915 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.600891 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-eeb61d-openshift-default-dockercfg-wtr55\"" Jul 16 00:46:43.611731 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.611703 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg"] Jul 16 00:46:43.746658 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.746617 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7tfd\" (UniqueName: \"kubernetes.io/projected/e3619251-9305-465b-8610-2b1a0642abf6-kube-api-access-j7tfd\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.746847 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.746702 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3619251-9305-465b-8610-2b1a0642abf6-istiod-ca-cert\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.746847 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.746728 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-credential-socket\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.746847 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.746758 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3619251-9305-465b-8610-2b1a0642abf6-istio-podinfo\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.746847 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.746831 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-workload-certs\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.747015 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.746857 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3619251-9305-465b-8610-2b1a0642abf6-istio-token\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.747015 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.746902 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-istio-envoy\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.747015 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.746990 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-workload-socket\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.747120 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.747027 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-istio-data\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.847743 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.847653 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3619251-9305-465b-8610-2b1a0642abf6-istiod-ca-cert\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.847743 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.847706 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-credential-socket\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.847743 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.847728 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3619251-9305-465b-8610-2b1a0642abf6-istio-podinfo\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.848030 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.847843 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-workload-certs\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.848030 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.847880 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3619251-9305-465b-8610-2b1a0642abf6-istio-token\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.848030 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.847952 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-istio-envoy\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.848030 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.848011 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-workload-socket\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.848215 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.848042 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-istio-data\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.848262 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.848222 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-credential-socket\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.848315 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.848291 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7tfd\" (UniqueName: \"kubernetes.io/projected/e3619251-9305-465b-8610-2b1a0642abf6-kube-api-access-j7tfd\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.848374 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.848334 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-workload-certs\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.848474 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.848455 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-istio-data\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.848529 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.848460 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3619251-9305-465b-8610-2b1a0642abf6-istiod-ca-cert\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.848619 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.848599 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-workload-socket\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.850194 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.850170 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-istio-envoy\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.850299 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.850279 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3619251-9305-465b-8610-2b1a0642abf6-istio-podinfo\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.856657 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.856639 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3619251-9305-465b-8610-2b1a0642abf6-istio-token\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.856906 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.856888 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7tfd\" (UniqueName: \"kubernetes.io/projected/e3619251-9305-465b-8610-2b1a0642abf6-kube-api-access-j7tfd\") pod \"e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:43.907849 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:43.907817 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:44.050303 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:44.050272 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg"] Jul 16 00:46:44.052626 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:46:44.052595 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3619251_9305_465b_8610_2b1a0642abf6.slice/crio-d7b50bda11561f70aade1102074a93f289e118db6b4a4d80f797b2ca8ca438e5 WatchSource:0}: Error finding container d7b50bda11561f70aade1102074a93f289e118db6b4a4d80f797b2ca8ca438e5: Status 404 returned error can't find the container with id d7b50bda11561f70aade1102074a93f289e118db6b4a4d80f797b2ca8ca438e5 Jul 16 00:46:44.054766 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:44.054735 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:46:44.054844 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:44.054799 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:46:44.054844 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:44.054824 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:46:44.149849 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:44.149807 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" event={"ID":"e3619251-9305-465b-8610-2b1a0642abf6","Type":"ContainerStarted","Data":"99d3a41888a55c15fac3fed9b08bedbd6219fc42a7b6989bfa05d923b4e92226"} Jul 16 00:46:44.150005 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:44.149877 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" event={"ID":"e3619251-9305-465b-8610-2b1a0642abf6","Type":"ContainerStarted","Data":"d7b50bda11561f70aade1102074a93f289e118db6b4a4d80f797b2ca8ca438e5"} Jul 16 00:46:44.180884 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:44.180818 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" podStartSLOduration=1.180796686 podStartE2EDuration="1.180796686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:46:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:46:44.178364239 +0000 UTC m=+2013.246304710" watchObservedRunningTime="2026-07-16 00:46:44.180796686 +0000 UTC m=+2013.248737162" Jul 16 00:46:44.908862 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:44.908827 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:45.913549 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:45.913515 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:46.158831 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:46.158800 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:46.159645 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:46.159626 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:47.942668 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:47.942626 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:46:56.211191 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:56.211156 2564 generic.go:356] "Generic (PLEG): container finished" podID="e3619251-9305-465b-8610-2b1a0642abf6" containerID="99d3a41888a55c15fac3fed9b08bedbd6219fc42a7b6989bfa05d923b4e92226" exitCode=0 Jul 16 00:46:56.211615 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:56.211234 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" event={"ID":"e3619251-9305-465b-8610-2b1a0642abf6","Type":"ContainerDied","Data":"99d3a41888a55c15fac3fed9b08bedbd6219fc42a7b6989bfa05d923b4e92226"} Jul 16 00:46:56.211661 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:56.211638 2564 scope.go:117] "RemoveContainer" containerID="99d3a41888a55c15fac3fed9b08bedbd6219fc42a7b6989bfa05d923b4e92226" Jul 16 00:46:56.212612 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:56.212589 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:46:56.212671 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:56.212654 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:46:56.212706 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:56.212692 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:46:57.218184 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:57.218145 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" event={"ID":"e3619251-9305-465b-8610-2b1a0642abf6","Type":"ContainerStarted","Data":"def45c69400fb04bcadf186521b398ac43bd272fb86b4f92d0df679a2337099b"} Jul 16 00:46:57.908240 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:57.908178 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:46:57.909323 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:57.909294 2564 patch_prober.go:28] interesting pod/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 16 00:46:57.909436 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:57.909359 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" podUID="e3619251-9305-465b-8610-2b1a0642abf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 16 00:46:58.908607 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:58.908570 2564 patch_prober.go:28] interesting pod/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 16 00:46:58.908993 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:58.908659 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" podUID="e3619251-9305-465b-8610-2b1a0642abf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 16 00:46:59.908290 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:59.908257 2564 patch_prober.go:28] interesting pod/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 16 00:46:59.908459 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:46:59.908310 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" podUID="e3619251-9305-465b-8610-2b1a0642abf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 16 00:47:00.912641 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:00.912612 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:47:00.913117 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:00.913003 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:47:00.914010 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:00.913988 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:47:08.045460 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:08.045424 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:47:20.543590 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:20.543551 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:47:22.269350 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.269319 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp"] Jul 16 00:47:22.272593 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.272576 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.275983 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.275956 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-eeb61d\"/\"kube-root-ca.crt\"" Jul 16 00:47:22.276867 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.276849 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-eeb61d\"/\"default-dockercfg-h9rrh\"" Jul 16 00:47:22.276952 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.276934 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-eeb61d\"/\"test-model-eeb61d-kserve-self-signed-certs\"" Jul 16 00:47:22.276997 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.276963 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-eeb61d\"/\"openshift-service-ca.crt\"" Jul 16 00:47:22.287429 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.287401 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp"] Jul 16 00:47:22.412235 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.412204 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-home\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.412446 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.412246 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-dshm\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.412446 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.412276 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-tmp-dir\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.412446 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.412346 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz9qq\" (UniqueName: \"kubernetes.io/projected/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-kube-api-access-bz9qq\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.412446 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.412409 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-model-cache\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.412649 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.412461 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-tls-certs\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.513723 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.513688 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz9qq\" (UniqueName: \"kubernetes.io/projected/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-kube-api-access-bz9qq\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.513893 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.513737 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-model-cache\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.513893 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.513772 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-tls-certs\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.513893 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.513815 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-home\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.513893 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.513837 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-dshm\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.513893 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.513857 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-tmp-dir\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.514222 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.514198 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-model-cache\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.514222 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.514215 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-home\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.514287 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.514269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-tmp-dir\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.516080 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.516059 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-dshm\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.516326 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.516309 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-tls-certs\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.525846 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.525757 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz9qq\" (UniqueName: \"kubernetes.io/projected/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-kube-api-access-bz9qq\") pod \"test-model-eeb61d-kserve-7688dc6cdc-npdpp\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.582321 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.582285 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:22.712723 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:22.712691 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp"] Jul 16 00:47:22.714890 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:47:22.714863 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2258c5a_7622_4e7a_9c0a_ac04e2624dce.slice/crio-df4923f18e45247556e3d8c183a30de038fa0eafdbdcadb80441db5e0d974bb7 WatchSource:0}: Error finding container df4923f18e45247556e3d8c183a30de038fa0eafdbdcadb80441db5e0d974bb7: Status 404 returned error can't find the container with id df4923f18e45247556e3d8c183a30de038fa0eafdbdcadb80441db5e0d974bb7 Jul 16 00:47:23.341948 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:23.341916 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" event={"ID":"f2258c5a-7622-4e7a-9c0a-ac04e2624dce","Type":"ContainerStarted","Data":"88f56cf26cc29ccfcc37d92c91964aa4ededb8d744a020abc7bccaf396427129"} Jul 16 00:47:23.342352 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:23.341956 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" event={"ID":"f2258c5a-7622-4e7a-9c0a-ac04e2624dce","Type":"ContainerStarted","Data":"df4923f18e45247556e3d8c183a30de038fa0eafdbdcadb80441db5e0d974bb7"} Jul 16 00:47:23.360795 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:23.360744 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" podStartSLOduration=1.360728472 podStartE2EDuration="1.360728472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:47:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:47:23.359892399 +0000 UTC m=+2052.427832872" watchObservedRunningTime="2026-07-16 00:47:23.360728472 +0000 UTC m=+2052.428668987" Jul 16 00:47:32.582771 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:32.582692 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:32.582771 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:32.582742 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:32.594658 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:32.594632 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:33.396616 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:33.396588 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:47:36.097519 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:36.097481 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:47:45.296904 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:45.296869 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp"] Jul 16 00:47:51.051453 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:47:51.051416 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:48:00.316359 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.316296 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" podUID="f2258c5a-7622-4e7a-9c0a-ac04e2624dce" containerName="main" containerID="cri-o://88f56cf26cc29ccfcc37d92c91964aa4ededb8d744a020abc7bccaf396427129" gracePeriod=45 Jul 16 00:48:00.516637 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.516605 2564 generic.go:356] "Generic (PLEG): container finished" podID="f2258c5a-7622-4e7a-9c0a-ac04e2624dce" containerID="88f56cf26cc29ccfcc37d92c91964aa4ededb8d744a020abc7bccaf396427129" exitCode=0 Jul 16 00:48:00.516790 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.516690 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" event={"ID":"f2258c5a-7622-4e7a-9c0a-ac04e2624dce","Type":"ContainerDied","Data":"88f56cf26cc29ccfcc37d92c91964aa4ededb8d744a020abc7bccaf396427129"} Jul 16 00:48:00.568066 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.568006 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:48:00.665140 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.665104 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-tls-certs\") pod \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " Jul 16 00:48:00.665313 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.665164 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-tmp-dir\") pod \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " Jul 16 00:48:00.665313 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.665190 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-home\") pod \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " Jul 16 00:48:00.665313 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.665210 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-dshm\") pod \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " Jul 16 00:48:00.665313 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.665236 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz9qq\" (UniqueName: \"kubernetes.io/projected/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-kube-api-access-bz9qq\") pod \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " Jul 16 00:48:00.665492 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.665329 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-model-cache\") pod \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\" (UID: \"f2258c5a-7622-4e7a-9c0a-ac04e2624dce\") " Jul 16 00:48:00.665614 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.665486 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-home" (OuterVolumeSpecName: "home") pod "f2258c5a-7622-4e7a-9c0a-ac04e2624dce" (UID: "f2258c5a-7622-4e7a-9c0a-ac04e2624dce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:48:00.665614 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.665500 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f2258c5a-7622-4e7a-9c0a-ac04e2624dce" (UID: "f2258c5a-7622-4e7a-9c0a-ac04e2624dce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:48:00.665774 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.665744 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-model-cache" (OuterVolumeSpecName: "model-cache") pod "f2258c5a-7622-4e7a-9c0a-ac04e2624dce" (UID: "f2258c5a-7622-4e7a-9c0a-ac04e2624dce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:48:00.665846 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.665776 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-tmp-dir\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:00.665846 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.665799 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-home\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:00.667474 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.667434 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-dshm" (OuterVolumeSpecName: "dshm") pod "f2258c5a-7622-4e7a-9c0a-ac04e2624dce" (UID: "f2258c5a-7622-4e7a-9c0a-ac04e2624dce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:48:00.667474 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.667440 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-kube-api-access-bz9qq" (OuterVolumeSpecName: "kube-api-access-bz9qq") pod "f2258c5a-7622-4e7a-9c0a-ac04e2624dce" (UID: "f2258c5a-7622-4e7a-9c0a-ac04e2624dce"). InnerVolumeSpecName "kube-api-access-bz9qq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:48:00.667596 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.667545 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f2258c5a-7622-4e7a-9c0a-ac04e2624dce" (UID: "f2258c5a-7622-4e7a-9c0a-ac04e2624dce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:48:00.767237 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.767201 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-tls-certs\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:00.767237 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.767230 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-dshm\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:00.767237 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.767243 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz9qq\" (UniqueName: \"kubernetes.io/projected/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-kube-api-access-bz9qq\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:00.767463 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:00.767257 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2258c5a-7622-4e7a-9c0a-ac04e2624dce-model-cache\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:01.522203 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:01.522176 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" Jul 16 00:48:01.522724 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:01.522174 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp" event={"ID":"f2258c5a-7622-4e7a-9c0a-ac04e2624dce","Type":"ContainerDied","Data":"df4923f18e45247556e3d8c183a30de038fa0eafdbdcadb80441db5e0d974bb7"} Jul 16 00:48:01.522724 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:01.522303 2564 scope.go:117] "RemoveContainer" containerID="88f56cf26cc29ccfcc37d92c91964aa4ededb8d744a020abc7bccaf396427129" Jul 16 00:48:01.548011 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:01.547987 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp"] Jul 16 00:48:01.558694 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:01.558663 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-eeb61d/test-model-eeb61d-kserve-7688dc6cdc-npdpp"] Jul 16 00:48:03.484957 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:03.484916 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2258c5a-7622-4e7a-9c0a-ac04e2624dce" path="/var/lib/kubelet/pods/f2258c5a-7622-4e7a-9c0a-ac04e2624dce/volumes" Jul 16 00:48:11.512278 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:11.512246 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:48:11.518945 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:11.518922 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:48:11.520095 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:11.520076 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:48:11.526514 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:11.526495 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:48:33.956115 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:33.956075 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:48:37.456173 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:37.456135 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg"] Jul 16 00:48:37.456638 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:37.456352 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" podUID="e3619251-9305-465b-8610-2b1a0642abf6" containerName="istio-proxy" containerID="cri-o://def45c69400fb04bcadf186521b398ac43bd272fb86b4f92d0df679a2337099b" gracePeriod=30 Jul 16 00:48:42.708465 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.708438 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:48:42.722860 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.722790 2564 generic.go:356] "Generic (PLEG): container finished" podID="e3619251-9305-465b-8610-2b1a0642abf6" containerID="def45c69400fb04bcadf186521b398ac43bd272fb86b4f92d0df679a2337099b" exitCode=0 Jul 16 00:48:42.722860 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.722845 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" Jul 16 00:48:42.722860 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.722850 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" event={"ID":"e3619251-9305-465b-8610-2b1a0642abf6","Type":"ContainerDied","Data":"def45c69400fb04bcadf186521b398ac43bd272fb86b4f92d0df679a2337099b"} Jul 16 00:48:42.723056 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.722913 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg" event={"ID":"e3619251-9305-465b-8610-2b1a0642abf6","Type":"ContainerDied","Data":"d7b50bda11561f70aade1102074a93f289e118db6b4a4d80f797b2ca8ca438e5"} Jul 16 00:48:42.723056 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.722933 2564 scope.go:117] "RemoveContainer" containerID="def45c69400fb04bcadf186521b398ac43bd272fb86b4f92d0df679a2337099b" Jul 16 00:48:42.730940 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.730921 2564 scope.go:117] "RemoveContainer" containerID="99d3a41888a55c15fac3fed9b08bedbd6219fc42a7b6989bfa05d923b4e92226" Jul 16 00:48:42.740005 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.739989 2564 scope.go:117] "RemoveContainer" containerID="def45c69400fb04bcadf186521b398ac43bd272fb86b4f92d0df679a2337099b" Jul 16 00:48:42.740301 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:48:42.740274 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"def45c69400fb04bcadf186521b398ac43bd272fb86b4f92d0df679a2337099b\": container with ID starting with def45c69400fb04bcadf186521b398ac43bd272fb86b4f92d0df679a2337099b not found: ID does not exist" containerID="def45c69400fb04bcadf186521b398ac43bd272fb86b4f92d0df679a2337099b" Jul 16 00:48:42.740356 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.740301 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"def45c69400fb04bcadf186521b398ac43bd272fb86b4f92d0df679a2337099b"} err="failed to get container status \"def45c69400fb04bcadf186521b398ac43bd272fb86b4f92d0df679a2337099b\": rpc error: code = NotFound desc = could not find container \"def45c69400fb04bcadf186521b398ac43bd272fb86b4f92d0df679a2337099b\": container with ID starting with def45c69400fb04bcadf186521b398ac43bd272fb86b4f92d0df679a2337099b not found: ID does not exist" Jul 16 00:48:42.740356 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.740322 2564 scope.go:117] "RemoveContainer" containerID="99d3a41888a55c15fac3fed9b08bedbd6219fc42a7b6989bfa05d923b4e92226" Jul 16 00:48:42.740611 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:48:42.740595 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99d3a41888a55c15fac3fed9b08bedbd6219fc42a7b6989bfa05d923b4e92226\": container with ID starting with 99d3a41888a55c15fac3fed9b08bedbd6219fc42a7b6989bfa05d923b4e92226 not found: ID does not exist" containerID="99d3a41888a55c15fac3fed9b08bedbd6219fc42a7b6989bfa05d923b4e92226" Jul 16 00:48:42.740670 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.740614 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99d3a41888a55c15fac3fed9b08bedbd6219fc42a7b6989bfa05d923b4e92226"} err="failed to get container status \"99d3a41888a55c15fac3fed9b08bedbd6219fc42a7b6989bfa05d923b4e92226\": rpc error: code = NotFound desc = could not find container \"99d3a41888a55c15fac3fed9b08bedbd6219fc42a7b6989bfa05d923b4e92226\": container with ID starting with 99d3a41888a55c15fac3fed9b08bedbd6219fc42a7b6989bfa05d923b4e92226 not found: ID does not exist" Jul 16 00:48:42.854216 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854178 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3619251-9305-465b-8610-2b1a0642abf6-istio-podinfo\") pod \"e3619251-9305-465b-8610-2b1a0642abf6\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " Jul 16 00:48:42.854216 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854221 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-credential-socket\") pod \"e3619251-9305-465b-8610-2b1a0642abf6\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " Jul 16 00:48:42.854404 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854242 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3619251-9305-465b-8610-2b1a0642abf6-istiod-ca-cert\") pod \"e3619251-9305-465b-8610-2b1a0642abf6\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " Jul 16 00:48:42.854404 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854268 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-istio-envoy\") pod \"e3619251-9305-465b-8610-2b1a0642abf6\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " Jul 16 00:48:42.854404 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854294 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-istio-data\") pod \"e3619251-9305-465b-8610-2b1a0642abf6\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " Jul 16 00:48:42.854404 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854389 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-workload-certs\") pod \"e3619251-9305-465b-8610-2b1a0642abf6\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " Jul 16 00:48:42.854624 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854418 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7tfd\" (UniqueName: \"kubernetes.io/projected/e3619251-9305-465b-8610-2b1a0642abf6-kube-api-access-j7tfd\") pod \"e3619251-9305-465b-8610-2b1a0642abf6\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " Jul 16 00:48:42.854624 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854472 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-workload-socket\") pod \"e3619251-9305-465b-8610-2b1a0642abf6\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " Jul 16 00:48:42.854624 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854501 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3619251-9305-465b-8610-2b1a0642abf6-istio-token\") pod \"e3619251-9305-465b-8610-2b1a0642abf6\" (UID: \"e3619251-9305-465b-8610-2b1a0642abf6\") " Jul 16 00:48:42.854624 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854481 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e3619251-9305-465b-8610-2b1a0642abf6" (UID: "e3619251-9305-465b-8610-2b1a0642abf6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:48:42.854878 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854726 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3619251-9305-465b-8610-2b1a0642abf6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e3619251-9305-465b-8610-2b1a0642abf6" (UID: "e3619251-9305-465b-8610-2b1a0642abf6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:48:42.854878 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854733 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e3619251-9305-465b-8610-2b1a0642abf6" (UID: "e3619251-9305-465b-8610-2b1a0642abf6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:48:42.854878 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854760 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e3619251-9305-465b-8610-2b1a0642abf6" (UID: "e3619251-9305-465b-8610-2b1a0642abf6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:48:42.855025 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854876 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-credential-socket\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:42.855025 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854895 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3619251-9305-465b-8610-2b1a0642abf6-istiod-ca-cert\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:42.855025 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854909 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-workload-certs\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:42.855025 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.854922 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-workload-socket\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:42.855377 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.855352 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-istio-data" (OuterVolumeSpecName: "istio-data") pod "e3619251-9305-465b-8610-2b1a0642abf6" (UID: "e3619251-9305-465b-8610-2b1a0642abf6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:48:42.856768 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.856742 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e3619251-9305-465b-8610-2b1a0642abf6" (UID: "e3619251-9305-465b-8610-2b1a0642abf6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:48:42.856904 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.856877 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3619251-9305-465b-8610-2b1a0642abf6-istio-token" (OuterVolumeSpecName: "istio-token") pod "e3619251-9305-465b-8610-2b1a0642abf6" (UID: "e3619251-9305-465b-8610-2b1a0642abf6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:48:42.856961 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.856903 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3619251-9305-465b-8610-2b1a0642abf6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e3619251-9305-465b-8610-2b1a0642abf6" (UID: "e3619251-9305-465b-8610-2b1a0642abf6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 00:48:42.857103 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.857081 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3619251-9305-465b-8610-2b1a0642abf6-kube-api-access-j7tfd" (OuterVolumeSpecName: "kube-api-access-j7tfd") pod "e3619251-9305-465b-8610-2b1a0642abf6" (UID: "e3619251-9305-465b-8610-2b1a0642abf6"). InnerVolumeSpecName "kube-api-access-j7tfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:48:42.955867 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.955835 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3619251-9305-465b-8610-2b1a0642abf6-istio-token\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:42.955867 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.955862 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3619251-9305-465b-8610-2b1a0642abf6-istio-podinfo\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:42.955867 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.955871 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-istio-envoy\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:42.956097 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.955879 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3619251-9305-465b-8610-2b1a0642abf6-istio-data\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:42.956097 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:42.955887 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7tfd\" (UniqueName: \"kubernetes.io/projected/e3619251-9305-465b-8610-2b1a0642abf6-kube-api-access-j7tfd\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:48:43.045793 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:43.045762 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg"] Jul 16 00:48:43.052422 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:43.052395 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-eeb61d-openshift-default-7fd77d8b7c-4tjlg"] Jul 16 00:48:43.485422 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:43.485340 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3619251-9305-465b-8610-2b1a0642abf6" path="/var/lib/kubelet/pods/e3619251-9305-465b-8610-2b1a0642abf6/volumes" Jul 16 00:48:47.747331 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:48:47.747287 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:49:01.260596 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:01.260562 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:49:19.453295 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:19.453259 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:49:29.407681 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.407651 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46"] Jul 16 00:49:29.408060 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.407990 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3619251-9305-465b-8610-2b1a0642abf6" containerName="istio-proxy" Jul 16 00:49:29.408060 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.408001 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3619251-9305-465b-8610-2b1a0642abf6" containerName="istio-proxy" Jul 16 00:49:29.408060 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.408014 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2258c5a-7622-4e7a-9c0a-ac04e2624dce" containerName="main" Jul 16 00:49:29.408060 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.408020 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2258c5a-7622-4e7a-9c0a-ac04e2624dce" containerName="main" Jul 16 00:49:29.408186 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.408066 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3619251-9305-465b-8610-2b1a0642abf6" containerName="istio-proxy" Jul 16 00:49:29.408186 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.408077 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2258c5a-7622-4e7a-9c0a-ac04e2624dce" containerName="main" Jul 16 00:49:29.410998 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.410981 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.413935 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.413908 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-7ee350-openshift-default-dockercfg-vz75r\"" Jul 16 00:49:29.431035 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.431010 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46"] Jul 16 00:49:29.576441 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.576402 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-token\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.576645 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.576472 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-podinfo\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.576645 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.576509 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec9a7008-5e58-4c36-b618-c2d808660e9c-istiod-ca-cert\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.576645 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.576561 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-envoy\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.576645 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.576587 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22hwm\" (UniqueName: \"kubernetes.io/projected/ec9a7008-5e58-4c36-b618-c2d808660e9c-kube-api-access-22hwm\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.576877 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.576651 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-workload-socket\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.576877 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.576694 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-data\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.576963 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.576878 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-credential-socket\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.576963 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.576902 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-workload-certs\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.677385 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.677289 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-podinfo\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.677385 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.677331 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec9a7008-5e58-4c36-b618-c2d808660e9c-istiod-ca-cert\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.677385 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.677350 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-envoy\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.677385 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.677365 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22hwm\" (UniqueName: \"kubernetes.io/projected/ec9a7008-5e58-4c36-b618-c2d808660e9c-kube-api-access-22hwm\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.677385 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.677390 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-workload-socket\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.677795 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.677490 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-data\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.677795 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.677529 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-credential-socket\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.677795 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.677578 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-workload-certs\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.677795 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.677631 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-token\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.678001 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.677890 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-data\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.678001 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.677931 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-workload-socket\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.678001 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.677959 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-credential-socket\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.678163 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.678146 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-workload-certs\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.678234 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.678206 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec9a7008-5e58-4c36-b618-c2d808660e9c-istiod-ca-cert\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.679770 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.679748 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-envoy\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.680167 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.680150 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-podinfo\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.686748 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.686724 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-token\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.686862 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.686846 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22hwm\" (UniqueName: \"kubernetes.io/projected/ec9a7008-5e58-4c36-b618-c2d808660e9c-kube-api-access-22hwm\") pod \"e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.720798 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.720765 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:29.858769 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.858731 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46"] Jul 16 00:49:29.860711 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:49:29.860677 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec9a7008_5e58_4c36_b618_c2d808660e9c.slice/crio-790b48bb3f0419adbcc9b9dca7ec0402f3b6a9c1b02c319085a2340d4a62d814 WatchSource:0}: Error finding container 790b48bb3f0419adbcc9b9dca7ec0402f3b6a9c1b02c319085a2340d4a62d814: Status 404 returned error can't find the container with id 790b48bb3f0419adbcc9b9dca7ec0402f3b6a9c1b02c319085a2340d4a62d814 Jul 16 00:49:29.863117 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.863083 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:49:29.863207 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.863152 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:49:29.863207 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.863180 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:49:29.934937 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:29.934732 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" event={"ID":"ec9a7008-5e58-4c36-b618-c2d808660e9c","Type":"ContainerStarted","Data":"790b48bb3f0419adbcc9b9dca7ec0402f3b6a9c1b02c319085a2340d4a62d814"} Jul 16 00:49:30.941209 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:30.941174 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" event={"ID":"ec9a7008-5e58-4c36-b618-c2d808660e9c","Type":"ContainerStarted","Data":"6a6c633cc4e8aea21e274864d1738696ff0ecf40b1898cca168e2db171c84dc0"} Jul 16 00:49:30.965221 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:30.965165 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" podStartSLOduration=1.965147645 podStartE2EDuration="1.965147645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:49:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:49:30.964635109 +0000 UTC m=+2180.032575585" watchObservedRunningTime="2026-07-16 00:49:30.965147645 +0000 UTC m=+2180.033088120" Jul 16 00:49:31.721884 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:31.721846 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:31.726637 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:31.726611 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:31.945837 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:31.945802 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:31.947039 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:31.947014 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:34.849489 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:34.849456 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:49:43.003175 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:43.003061 2564 generic.go:356] "Generic (PLEG): container finished" podID="ec9a7008-5e58-4c36-b618-c2d808660e9c" containerID="6a6c633cc4e8aea21e274864d1738696ff0ecf40b1898cca168e2db171c84dc0" exitCode=0 Jul 16 00:49:43.003175 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:43.003137 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" event={"ID":"ec9a7008-5e58-4c36-b618-c2d808660e9c","Type":"ContainerDied","Data":"6a6c633cc4e8aea21e274864d1738696ff0ecf40b1898cca168e2db171c84dc0"} Jul 16 00:49:43.003683 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:43.003526 2564 scope.go:117] "RemoveContainer" containerID="6a6c633cc4e8aea21e274864d1738696ff0ecf40b1898cca168e2db171c84dc0" Jul 16 00:49:43.004564 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:43.004522 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:49:43.004668 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:43.004602 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:49:43.004668 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:43.004656 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 00:49:44.010089 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:44.010054 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" event={"ID":"ec9a7008-5e58-4c36-b618-c2d808660e9c","Type":"ContainerStarted","Data":"ad1782dde7ad26298c421a16637f18e4f230e2a7e9a1670bb217597defe2fc0a"} Jul 16 00:49:44.721816 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:44.721777 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:44.723016 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:44.722987 2564 patch_prober.go:28] interesting pod/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 16 00:49:44.723139 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:44.723039 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" podUID="ec9a7008-5e58-4c36-b618-c2d808660e9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 16 00:49:45.721737 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:45.721700 2564 patch_prober.go:28] interesting pod/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 16 00:49:45.722146 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:45.721752 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" podUID="ec9a7008-5e58-4c36-b618-c2d808660e9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 16 00:49:46.721397 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:46.721354 2564 patch_prober.go:28] interesting pod/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 16 00:49:46.721608 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:46.721431 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" podUID="ec9a7008-5e58-4c36-b618-c2d808660e9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 16 00:49:46.946604 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:46.946563 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:47.721641 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:47.721603 2564 patch_prober.go:28] interesting pod/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 16 00:49:47.721816 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:47.721659 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" podUID="ec9a7008-5e58-4c36-b618-c2d808660e9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 16 00:49:47.748101 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:47.748063 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:49:48.724833 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:48.724803 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:49:48.725650 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:49:48.725631 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:50:03.148835 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:50:03.148801 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:50:33.561467 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:50:33.561375 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:50:53.463721 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:50:53.463685 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:51:14.950002 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:51:14.949964 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:51:31.843100 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:51:31.843064 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:51:49.841119 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:51:49.841079 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:52:12.035212 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:12.035130 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:52:32.772340 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:32.772302 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:52:53.657047 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:53.657012 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:52:55.968601 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:55.968568 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r"] Jul 16 00:52:55.969102 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:55.969083 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3619251-9305-465b-8610-2b1a0642abf6" containerName="istio-proxy" Jul 16 00:52:55.969180 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:55.969106 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3619251-9305-465b-8610-2b1a0642abf6" containerName="istio-proxy" Jul 16 00:52:55.969230 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:55.969197 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3619251-9305-465b-8610-2b1a0642abf6" containerName="istio-proxy" Jul 16 00:52:55.972167 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:55.972138 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:55.976787 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:55.976714 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7ee350\"/\"default-dockercfg-dzxw7\"" Jul 16 00:52:55.976787 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:55.976727 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7ee350\"/\"openshift-service-ca.crt\"" Jul 16 00:52:55.977032 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:55.976728 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7ee350\"/\"sub-test-model-7ee350-kserve-self-signed-certs\"" Jul 16 00:52:55.977378 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:55.977362 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7ee350\"/\"kube-root-ca.crt\"" Jul 16 00:52:55.986220 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:55.986193 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r"] Jul 16 00:52:56.082519 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.082484 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-model-cache\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.082699 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.082556 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-tmp-dir\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.082699 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.082587 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12e12554-d1a3-4f0d-9d0e-8c23bd159623-tls-certs\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.082699 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.082605 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-dshm\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.082699 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.082663 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-home\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.082699 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.082691 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s25b\" (UniqueName: \"kubernetes.io/projected/12e12554-d1a3-4f0d-9d0e-8c23bd159623-kube-api-access-7s25b\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.184013 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.183978 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12e12554-d1a3-4f0d-9d0e-8c23bd159623-tls-certs\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.184013 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.184019 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-dshm\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.184208 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.184145 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-home\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.184208 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.184196 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s25b\" (UniqueName: \"kubernetes.io/projected/12e12554-d1a3-4f0d-9d0e-8c23bd159623-kube-api-access-7s25b\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.184349 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.184329 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-model-cache\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.184428 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.184363 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-tmp-dir\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.184614 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.184567 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-home\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.184762 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.184737 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-model-cache\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.184932 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.184751 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-tmp-dir\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.186709 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.186662 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-dshm\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.187062 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.186902 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12e12554-d1a3-4f0d-9d0e-8c23bd159623-tls-certs\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.197846 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.197149 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s25b\" (UniqueName: \"kubernetes.io/projected/12e12554-d1a3-4f0d-9d0e-8c23bd159623-kube-api-access-7s25b\") pod \"sub-test-model-7ee350-kserve-78554b5c45-zfl8r\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.281601 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.281566 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:52:56.422054 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.421996 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r"] Jul 16 00:52:56.426704 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:52:56.426677 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12e12554_d1a3_4f0d_9d0e_8c23bd159623.slice/crio-4922b11b973cff0f5f6b1450653e526121408ea666fb694b532bc5c6c12946ae WatchSource:0}: Error finding container 4922b11b973cff0f5f6b1450653e526121408ea666fb694b532bc5c6c12946ae: Status 404 returned error can't find the container with id 4922b11b973cff0f5f6b1450653e526121408ea666fb694b532bc5c6c12946ae Jul 16 00:52:56.914689 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.914645 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" event={"ID":"12e12554-d1a3-4f0d-9d0e-8c23bd159623","Type":"ContainerStarted","Data":"9074e56416871d80981e4b8b7b51ba4e20c7161a6f17ac79f734c9accb718aed"} Jul 16 00:52:56.914871 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.914696 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" event={"ID":"12e12554-d1a3-4f0d-9d0e-8c23bd159623","Type":"ContainerStarted","Data":"4922b11b973cff0f5f6b1450653e526121408ea666fb694b532bc5c6c12946ae"} Jul 16 00:52:56.935945 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:52:56.935882 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" podStartSLOduration=1.9358644040000001 podStartE2EDuration="1.935864404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:52:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:52:56.933857772 +0000 UTC m=+2386.001798246" watchObservedRunningTime="2026-07-16 00:52:56.935864404 +0000 UTC m=+2386.003804899" Jul 16 00:53:06.282070 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:53:06.282024 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:53:06.282605 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:53:06.282084 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:53:06.294384 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:53:06.294359 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:53:06.971473 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:53:06.971445 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:53:11.548133 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:53:11.548104 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:53:11.554724 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:53:11.554691 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:53:11.555966 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:53:11.555941 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:53:11.562502 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:53:11.562483 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:53:13.037510 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:53:13.037474 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:53:30.061320 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:53:30.061281 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:53:49.140350 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:53:49.140252 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:53:54.402194 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:53:54.402147 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r"] Jul 16 00:54:08.362757 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:08.362721 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:54:09.422449 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.422380 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" podUID="12e12554-d1a3-4f0d-9d0e-8c23bd159623" containerName="main" containerID="cri-o://9074e56416871d80981e4b8b7b51ba4e20c7161a6f17ac79f734c9accb718aed" gracePeriod=45 Jul 16 00:54:09.676271 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.676214 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:54:09.756043 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.756016 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-dshm\") pod \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " Jul 16 00:54:09.756202 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.756095 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s25b\" (UniqueName: \"kubernetes.io/projected/12e12554-d1a3-4f0d-9d0e-8c23bd159623-kube-api-access-7s25b\") pod \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " Jul 16 00:54:09.756202 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.756121 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-model-cache\") pod \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " Jul 16 00:54:09.756202 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.756182 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-tmp-dir\") pod \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " Jul 16 00:54:09.756202 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.756201 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-home\") pod \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " Jul 16 00:54:09.756410 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.756224 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12e12554-d1a3-4f0d-9d0e-8c23bd159623-tls-certs\") pod \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\" (UID: \"12e12554-d1a3-4f0d-9d0e-8c23bd159623\") " Jul 16 00:54:09.756554 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.756429 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "12e12554-d1a3-4f0d-9d0e-8c23bd159623" (UID: "12e12554-d1a3-4f0d-9d0e-8c23bd159623"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:54:09.756554 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.756477 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-model-cache" (OuterVolumeSpecName: "model-cache") pod "12e12554-d1a3-4f0d-9d0e-8c23bd159623" (UID: "12e12554-d1a3-4f0d-9d0e-8c23bd159623"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:54:09.756641 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.756516 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-home" (OuterVolumeSpecName: "home") pod "12e12554-d1a3-4f0d-9d0e-8c23bd159623" (UID: "12e12554-d1a3-4f0d-9d0e-8c23bd159623"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:54:09.758300 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.758268 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12e12554-d1a3-4f0d-9d0e-8c23bd159623-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "12e12554-d1a3-4f0d-9d0e-8c23bd159623" (UID: "12e12554-d1a3-4f0d-9d0e-8c23bd159623"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:54:09.758401 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.758304 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-dshm" (OuterVolumeSpecName: "dshm") pod "12e12554-d1a3-4f0d-9d0e-8c23bd159623" (UID: "12e12554-d1a3-4f0d-9d0e-8c23bd159623"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:54:09.758401 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.758326 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12e12554-d1a3-4f0d-9d0e-8c23bd159623-kube-api-access-7s25b" (OuterVolumeSpecName: "kube-api-access-7s25b") pod "12e12554-d1a3-4f0d-9d0e-8c23bd159623" (UID: "12e12554-d1a3-4f0d-9d0e-8c23bd159623"). InnerVolumeSpecName "kube-api-access-7s25b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:54:09.857054 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.857015 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-dshm\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:54:09.857054 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.857047 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7s25b\" (UniqueName: \"kubernetes.io/projected/12e12554-d1a3-4f0d-9d0e-8c23bd159623-kube-api-access-7s25b\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:54:09.857054 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.857061 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-model-cache\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:54:09.857276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.857070 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-tmp-dir\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:54:09.857276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.857078 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12e12554-d1a3-4f0d-9d0e-8c23bd159623-home\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:54:09.857276 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:09.857086 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12e12554-d1a3-4f0d-9d0e-8c23bd159623-tls-certs\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:54:10.254210 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:10.254177 2564 generic.go:356] "Generic (PLEG): container finished" podID="12e12554-d1a3-4f0d-9d0e-8c23bd159623" containerID="9074e56416871d80981e4b8b7b51ba4e20c7161a6f17ac79f734c9accb718aed" exitCode=0 Jul 16 00:54:10.254391 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:10.254230 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" Jul 16 00:54:10.254391 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:10.254217 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" event={"ID":"12e12554-d1a3-4f0d-9d0e-8c23bd159623","Type":"ContainerDied","Data":"9074e56416871d80981e4b8b7b51ba4e20c7161a6f17ac79f734c9accb718aed"} Jul 16 00:54:10.254391 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:10.254306 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r" event={"ID":"12e12554-d1a3-4f0d-9d0e-8c23bd159623","Type":"ContainerDied","Data":"4922b11b973cff0f5f6b1450653e526121408ea666fb694b532bc5c6c12946ae"} Jul 16 00:54:10.254391 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:10.254327 2564 scope.go:117] "RemoveContainer" containerID="9074e56416871d80981e4b8b7b51ba4e20c7161a6f17ac79f734c9accb718aed" Jul 16 00:54:10.263323 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:10.263298 2564 scope.go:117] "RemoveContainer" containerID="9074e56416871d80981e4b8b7b51ba4e20c7161a6f17ac79f734c9accb718aed" Jul 16 00:54:10.263596 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:54:10.263574 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9074e56416871d80981e4b8b7b51ba4e20c7161a6f17ac79f734c9accb718aed\": container with ID starting with 9074e56416871d80981e4b8b7b51ba4e20c7161a6f17ac79f734c9accb718aed not found: ID does not exist" containerID="9074e56416871d80981e4b8b7b51ba4e20c7161a6f17ac79f734c9accb718aed" Jul 16 00:54:10.263641 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:10.263609 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9074e56416871d80981e4b8b7b51ba4e20c7161a6f17ac79f734c9accb718aed"} err="failed to get container status \"9074e56416871d80981e4b8b7b51ba4e20c7161a6f17ac79f734c9accb718aed\": rpc error: code = NotFound desc = could not find container \"9074e56416871d80981e4b8b7b51ba4e20c7161a6f17ac79f734c9accb718aed\": container with ID starting with 9074e56416871d80981e4b8b7b51ba4e20c7161a6f17ac79f734c9accb718aed not found: ID does not exist" Jul 16 00:54:10.276035 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:10.276002 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r"] Jul 16 00:54:10.282293 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:10.282269 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7ee350/sub-test-model-7ee350-kserve-78554b5c45-zfl8r"] Jul 16 00:54:11.484853 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:11.484822 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12e12554-d1a3-4f0d-9d0e-8c23bd159623" path="/var/lib/kubelet/pods/12e12554-d1a3-4f0d-9d0e-8c23bd159623/volumes" Jul 16 00:54:35.063753 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.063716 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj"] Jul 16 00:54:35.064226 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.064100 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12e12554-d1a3-4f0d-9d0e-8c23bd159623" containerName="main" Jul 16 00:54:35.064226 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.064112 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="12e12554-d1a3-4f0d-9d0e-8c23bd159623" containerName="main" Jul 16 00:54:35.064226 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.064181 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="12e12554-d1a3-4f0d-9d0e-8c23bd159623" containerName="main" Jul 16 00:54:35.067131 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.067115 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.070911 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.070842 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-07a8dc\"/\"rate-test-model-07a8dc-kserve-self-signed-certs\"" Jul 16 00:54:35.071073 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.070913 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-07a8dc\"/\"openshift-service-ca.crt\"" Jul 16 00:54:35.071073 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.071015 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-07a8dc\"/\"kube-root-ca.crt\"" Jul 16 00:54:35.071247 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.071223 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-07a8dc\"/\"default-dockercfg-ckt2l\"" Jul 16 00:54:35.083499 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.083468 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj"] Jul 16 00:54:35.196205 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.196165 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfqp6\" (UniqueName: \"kubernetes.io/projected/ac88e9c0-e318-47af-ae14-5341dc95892f-kube-api-access-qfqp6\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.196438 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.196215 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac88e9c0-e318-47af-ae14-5341dc95892f-tls-certs\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.196438 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.196277 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-model-cache\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.196438 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.196311 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-home\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.196438 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.196335 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-dshm\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.196438 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.196377 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-tmp-dir\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.297447 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.297406 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqp6\" (UniqueName: \"kubernetes.io/projected/ac88e9c0-e318-47af-ae14-5341dc95892f-kube-api-access-qfqp6\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.297642 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.297459 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac88e9c0-e318-47af-ae14-5341dc95892f-tls-certs\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.297642 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.297497 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-model-cache\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.297642 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.297526 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-home\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.297642 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.297591 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-dshm\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.297813 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.297718 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-tmp-dir\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.297961 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.297944 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-model-cache\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.298036 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.297994 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-home\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.298160 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.298121 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-tmp-dir\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.299909 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.299884 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-dshm\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.300208 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.300188 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac88e9c0-e318-47af-ae14-5341dc95892f-tls-certs\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.306268 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.306244 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfqp6\" (UniqueName: \"kubernetes.io/projected/ac88e9c0-e318-47af-ae14-5341dc95892f-kube-api-access-qfqp6\") pod \"rate-test-model-07a8dc-kserve-6975ff9799-6wpfj\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.377578 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.377463 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:35.512921 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:35.512897 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj"] Jul 16 00:54:35.515227 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:54:35.515202 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac88e9c0_e318_47af_ae14_5341dc95892f.slice/crio-9e95ecf5b1563fbbd4c2598ff024b83db395a4a19bf7af9fdb3dc1c4b698b57f WatchSource:0}: Error finding container 9e95ecf5b1563fbbd4c2598ff024b83db395a4a19bf7af9fdb3dc1c4b698b57f: Status 404 returned error can't find the container with id 9e95ecf5b1563fbbd4c2598ff024b83db395a4a19bf7af9fdb3dc1c4b698b57f Jul 16 00:54:36.378763 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:36.378729 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" event={"ID":"ac88e9c0-e318-47af-ae14-5341dc95892f","Type":"ContainerStarted","Data":"9bfcdf3a9dccb4c0a6539a0f056a271030a0889524b2a7b54c626d27375f192e"} Jul 16 00:54:36.378763 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:36.378773 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" event={"ID":"ac88e9c0-e318-47af-ae14-5341dc95892f","Type":"ContainerStarted","Data":"9e95ecf5b1563fbbd4c2598ff024b83db395a4a19bf7af9fdb3dc1c4b698b57f"} Jul 16 00:54:36.403124 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:36.403069 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" podStartSLOduration=1.403052735 podStartE2EDuration="1.403052735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 00:54:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:54:36.401243134 +0000 UTC m=+2485.469183617" watchObservedRunningTime="2026-07-16 00:54:36.403052735 +0000 UTC m=+2485.470993211" Jul 16 00:54:40.140909 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:40.140870 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:54:45.378034 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:45.377966 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:45.380615 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:45.378051 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:45.390753 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:45.390730 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:45.430048 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:45.430014 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:54:57.239869 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:54:57.239824 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:55:14.733894 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:14.733856 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:55:21.955216 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:21.955170 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj"] Jul 16 00:55:35.337491 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:35.337453 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:55:36.975209 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:36.975135 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" podUID="ac88e9c0-e318-47af-ae14-5341dc95892f" containerName="main" containerID="cri-o://9bfcdf3a9dccb4c0a6539a0f056a271030a0889524b2a7b54c626d27375f192e" gracePeriod=45 Jul 16 00:55:37.226167 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.226104 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:55:37.368069 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.368034 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-dshm\") pod \"ac88e9c0-e318-47af-ae14-5341dc95892f\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " Jul 16 00:55:37.368258 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.368114 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-tmp-dir\") pod \"ac88e9c0-e318-47af-ae14-5341dc95892f\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " Jul 16 00:55:37.368258 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.368161 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-home\") pod \"ac88e9c0-e318-47af-ae14-5341dc95892f\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " Jul 16 00:55:37.368258 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.368201 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac88e9c0-e318-47af-ae14-5341dc95892f-tls-certs\") pod \"ac88e9c0-e318-47af-ae14-5341dc95892f\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " Jul 16 00:55:37.368258 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.368255 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-model-cache\") pod \"ac88e9c0-e318-47af-ae14-5341dc95892f\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " Jul 16 00:55:37.368567 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.368295 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfqp6\" (UniqueName: \"kubernetes.io/projected/ac88e9c0-e318-47af-ae14-5341dc95892f-kube-api-access-qfqp6\") pod \"ac88e9c0-e318-47af-ae14-5341dc95892f\" (UID: \"ac88e9c0-e318-47af-ae14-5341dc95892f\") " Jul 16 00:55:37.368567 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.368337 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ac88e9c0-e318-47af-ae14-5341dc95892f" (UID: "ac88e9c0-e318-47af-ae14-5341dc95892f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:55:37.368567 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.368421 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-home" (OuterVolumeSpecName: "home") pod "ac88e9c0-e318-47af-ae14-5341dc95892f" (UID: "ac88e9c0-e318-47af-ae14-5341dc95892f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:55:37.368567 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.368548 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-model-cache" (OuterVolumeSpecName: "model-cache") pod "ac88e9c0-e318-47af-ae14-5341dc95892f" (UID: "ac88e9c0-e318-47af-ae14-5341dc95892f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:55:37.368743 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.368654 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-model-cache\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:37.368743 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.368675 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-tmp-dir\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:37.368743 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.368687 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-home\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:37.370279 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.370255 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-dshm" (OuterVolumeSpecName: "dshm") pod "ac88e9c0-e318-47af-ae14-5341dc95892f" (UID: "ac88e9c0-e318-47af-ae14-5341dc95892f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:55:37.370452 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.370434 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac88e9c0-e318-47af-ae14-5341dc95892f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ac88e9c0-e318-47af-ae14-5341dc95892f" (UID: "ac88e9c0-e318-47af-ae14-5341dc95892f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 00:55:37.370452 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.370437 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac88e9c0-e318-47af-ae14-5341dc95892f-kube-api-access-qfqp6" (OuterVolumeSpecName: "kube-api-access-qfqp6") pod "ac88e9c0-e318-47af-ae14-5341dc95892f" (UID: "ac88e9c0-e318-47af-ae14-5341dc95892f"). InnerVolumeSpecName "kube-api-access-qfqp6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:55:37.469834 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.469799 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac88e9c0-e318-47af-ae14-5341dc95892f-tls-certs\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:37.469834 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.469827 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfqp6\" (UniqueName: \"kubernetes.io/projected/ac88e9c0-e318-47af-ae14-5341dc95892f-kube-api-access-qfqp6\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:37.469834 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.469837 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac88e9c0-e318-47af-ae14-5341dc95892f-dshm\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:37.666876 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.666844 2564 generic.go:356] "Generic (PLEG): container finished" podID="ac88e9c0-e318-47af-ae14-5341dc95892f" containerID="9bfcdf3a9dccb4c0a6539a0f056a271030a0889524b2a7b54c626d27375f192e" exitCode=0 Jul 16 00:55:37.667041 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.666889 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" Jul 16 00:55:37.667041 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.666926 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" event={"ID":"ac88e9c0-e318-47af-ae14-5341dc95892f","Type":"ContainerDied","Data":"9bfcdf3a9dccb4c0a6539a0f056a271030a0889524b2a7b54c626d27375f192e"} Jul 16 00:55:37.667041 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.666968 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj" event={"ID":"ac88e9c0-e318-47af-ae14-5341dc95892f","Type":"ContainerDied","Data":"9e95ecf5b1563fbbd4c2598ff024b83db395a4a19bf7af9fdb3dc1c4b698b57f"} Jul 16 00:55:37.667041 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.666985 2564 scope.go:117] "RemoveContainer" containerID="9bfcdf3a9dccb4c0a6539a0f056a271030a0889524b2a7b54c626d27375f192e" Jul 16 00:55:37.675347 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.675324 2564 scope.go:117] "RemoveContainer" containerID="9bfcdf3a9dccb4c0a6539a0f056a271030a0889524b2a7b54c626d27375f192e" Jul 16 00:55:37.675682 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:55:37.675663 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bfcdf3a9dccb4c0a6539a0f056a271030a0889524b2a7b54c626d27375f192e\": container with ID starting with 9bfcdf3a9dccb4c0a6539a0f056a271030a0889524b2a7b54c626d27375f192e not found: ID does not exist" containerID="9bfcdf3a9dccb4c0a6539a0f056a271030a0889524b2a7b54c626d27375f192e" Jul 16 00:55:37.675754 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.675693 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bfcdf3a9dccb4c0a6539a0f056a271030a0889524b2a7b54c626d27375f192e"} err="failed to get container status \"9bfcdf3a9dccb4c0a6539a0f056a271030a0889524b2a7b54c626d27375f192e\": rpc error: code = NotFound desc = could not find container \"9bfcdf3a9dccb4c0a6539a0f056a271030a0889524b2a7b54c626d27375f192e\": container with ID starting with 9bfcdf3a9dccb4c0a6539a0f056a271030a0889524b2a7b54c626d27375f192e not found: ID does not exist" Jul 16 00:55:37.684478 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.684454 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj"] Jul 16 00:55:37.688236 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:37.688213 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-07a8dc/rate-test-model-07a8dc-kserve-6975ff9799-6wpfj"] Jul 16 00:55:39.485120 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:39.485084 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac88e9c0-e318-47af-ae14-5341dc95892f" path="/var/lib/kubelet/pods/ac88e9c0-e318-47af-ae14-5341dc95892f/volumes" Jul 16 00:55:48.293368 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:48.293333 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46"] Jul 16 00:55:48.293887 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:48.293591 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" podUID="ec9a7008-5e58-4c36-b618-c2d808660e9c" containerName="istio-proxy" containerID="cri-o://ad1782dde7ad26298c421a16637f18e4f230e2a7e9a1670bb217597defe2fc0a" gracePeriod=30 Jul 16 00:55:48.727223 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:48.727135 2564 patch_prober.go:28] interesting pod/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 00:55:48.727223 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:48.727211 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" podUID="ec9a7008-5e58-4c36-b618-c2d808660e9c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 00:55:52.634121 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:52.634087 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:55:53.541620 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.541593 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:55:53.714518 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.714425 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22hwm\" (UniqueName: \"kubernetes.io/projected/ec9a7008-5e58-4c36-b618-c2d808660e9c-kube-api-access-22hwm\") pod \"ec9a7008-5e58-4c36-b618-c2d808660e9c\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " Jul 16 00:55:53.714518 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.714475 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-workload-certs\") pod \"ec9a7008-5e58-4c36-b618-c2d808660e9c\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " Jul 16 00:55:53.715049 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.714665 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-token\") pod \"ec9a7008-5e58-4c36-b618-c2d808660e9c\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " Jul 16 00:55:53.715049 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.714733 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-podinfo\") pod \"ec9a7008-5e58-4c36-b618-c2d808660e9c\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " Jul 16 00:55:53.715049 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.714777 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-envoy\") pod \"ec9a7008-5e58-4c36-b618-c2d808660e9c\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " Jul 16 00:55:53.715049 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.714779 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ec9a7008-5e58-4c36-b618-c2d808660e9c" (UID: "ec9a7008-5e58-4c36-b618-c2d808660e9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:55:53.715049 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.714801 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-workload-socket\") pod \"ec9a7008-5e58-4c36-b618-c2d808660e9c\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " Jul 16 00:55:53.715049 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.714825 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-credential-socket\") pod \"ec9a7008-5e58-4c36-b618-c2d808660e9c\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " Jul 16 00:55:53.715049 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.714878 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec9a7008-5e58-4c36-b618-c2d808660e9c-istiod-ca-cert\") pod \"ec9a7008-5e58-4c36-b618-c2d808660e9c\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " Jul 16 00:55:53.715049 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.714905 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-data\") pod \"ec9a7008-5e58-4c36-b618-c2d808660e9c\" (UID: \"ec9a7008-5e58-4c36-b618-c2d808660e9c\") " Jul 16 00:55:53.715431 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.715049 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ec9a7008-5e58-4c36-b618-c2d808660e9c" (UID: "ec9a7008-5e58-4c36-b618-c2d808660e9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:55:53.715431 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.715089 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ec9a7008-5e58-4c36-b618-c2d808660e9c" (UID: "ec9a7008-5e58-4c36-b618-c2d808660e9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:55:53.715431 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.715289 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-workload-socket\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:53.715431 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.715310 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-credential-socket\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:53.715431 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.715325 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-workload-certs\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:53.715431 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.715424 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec9a7008-5e58-4c36-b618-c2d808660e9c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ec9a7008-5e58-4c36-b618-c2d808660e9c" (UID: "ec9a7008-5e58-4c36-b618-c2d808660e9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 00:55:53.715898 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.715873 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-data" (OuterVolumeSpecName: "istio-data") pod "ec9a7008-5e58-4c36-b618-c2d808660e9c" (UID: "ec9a7008-5e58-4c36-b618-c2d808660e9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:55:53.716843 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.716820 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec9a7008-5e58-4c36-b618-c2d808660e9c" (UID: "ec9a7008-5e58-4c36-b618-c2d808660e9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:55:53.716968 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.716917 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ec9a7008-5e58-4c36-b618-c2d808660e9c" (UID: "ec9a7008-5e58-4c36-b618-c2d808660e9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 00:55:53.716968 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.716924 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec9a7008-5e58-4c36-b618-c2d808660e9c-kube-api-access-22hwm" (OuterVolumeSpecName: "kube-api-access-22hwm") pod "ec9a7008-5e58-4c36-b618-c2d808660e9c" (UID: "ec9a7008-5e58-4c36-b618-c2d808660e9c"). InnerVolumeSpecName "kube-api-access-22hwm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 00:55:53.717045 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.716965 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ec9a7008-5e58-4c36-b618-c2d808660e9c" (UID: "ec9a7008-5e58-4c36-b618-c2d808660e9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 00:55:53.741446 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.741418 2564 generic.go:356] "Generic (PLEG): container finished" podID="ec9a7008-5e58-4c36-b618-c2d808660e9c" containerID="ad1782dde7ad26298c421a16637f18e4f230e2a7e9a1670bb217597defe2fc0a" exitCode=0 Jul 16 00:55:53.741600 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.741463 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" event={"ID":"ec9a7008-5e58-4c36-b618-c2d808660e9c","Type":"ContainerDied","Data":"ad1782dde7ad26298c421a16637f18e4f230e2a7e9a1670bb217597defe2fc0a"} Jul 16 00:55:53.741600 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.741470 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" Jul 16 00:55:53.741600 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.741503 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46" event={"ID":"ec9a7008-5e58-4c36-b618-c2d808660e9c","Type":"ContainerDied","Data":"790b48bb3f0419adbcc9b9dca7ec0402f3b6a9c1b02c319085a2340d4a62d814"} Jul 16 00:55:53.741600 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.741525 2564 scope.go:117] "RemoveContainer" containerID="ad1782dde7ad26298c421a16637f18e4f230e2a7e9a1670bb217597defe2fc0a" Jul 16 00:55:53.750807 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.750790 2564 scope.go:117] "RemoveContainer" containerID="6a6c633cc4e8aea21e274864d1738696ff0ecf40b1898cca168e2db171c84dc0" Jul 16 00:55:53.758500 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.758480 2564 scope.go:117] "RemoveContainer" containerID="ad1782dde7ad26298c421a16637f18e4f230e2a7e9a1670bb217597defe2fc0a" Jul 16 00:55:53.758760 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:55:53.758744 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad1782dde7ad26298c421a16637f18e4f230e2a7e9a1670bb217597defe2fc0a\": container with ID starting with ad1782dde7ad26298c421a16637f18e4f230e2a7e9a1670bb217597defe2fc0a not found: ID does not exist" containerID="ad1782dde7ad26298c421a16637f18e4f230e2a7e9a1670bb217597defe2fc0a" Jul 16 00:55:53.758801 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.758767 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1782dde7ad26298c421a16637f18e4f230e2a7e9a1670bb217597defe2fc0a"} err="failed to get container status \"ad1782dde7ad26298c421a16637f18e4f230e2a7e9a1670bb217597defe2fc0a\": rpc error: code = NotFound desc = could not find container \"ad1782dde7ad26298c421a16637f18e4f230e2a7e9a1670bb217597defe2fc0a\": container with ID starting with ad1782dde7ad26298c421a16637f18e4f230e2a7e9a1670bb217597defe2fc0a not found: ID does not exist" Jul 16 00:55:53.758801 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.758782 2564 scope.go:117] "RemoveContainer" containerID="6a6c633cc4e8aea21e274864d1738696ff0ecf40b1898cca168e2db171c84dc0" Jul 16 00:55:53.759053 ip-10-0-128-224 kubenswrapper[2564]: E0716 00:55:53.759035 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a6c633cc4e8aea21e274864d1738696ff0ecf40b1898cca168e2db171c84dc0\": container with ID starting with 6a6c633cc4e8aea21e274864d1738696ff0ecf40b1898cca168e2db171c84dc0 not found: ID does not exist" containerID="6a6c633cc4e8aea21e274864d1738696ff0ecf40b1898cca168e2db171c84dc0" Jul 16 00:55:53.759101 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.759058 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a6c633cc4e8aea21e274864d1738696ff0ecf40b1898cca168e2db171c84dc0"} err="failed to get container status \"6a6c633cc4e8aea21e274864d1738696ff0ecf40b1898cca168e2db171c84dc0\": rpc error: code = NotFound desc = could not find container \"6a6c633cc4e8aea21e274864d1738696ff0ecf40b1898cca168e2db171c84dc0\": container with ID starting with 6a6c633cc4e8aea21e274864d1738696ff0ecf40b1898cca168e2db171c84dc0 not found: ID does not exist" Jul 16 00:55:53.763022 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.762996 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46"] Jul 16 00:55:53.766608 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.766586 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-7ee350-openshift-default-7ff9874f5-crt46"] Jul 16 00:55:53.815924 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.815895 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-token\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:53.815924 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.815921 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-podinfo\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:53.815924 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.815930 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-envoy\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:53.816151 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.815942 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec9a7008-5e58-4c36-b618-c2d808660e9c-istiod-ca-cert\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:53.816151 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.815949 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec9a7008-5e58-4c36-b618-c2d808660e9c-istio-data\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:53.816151 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:53.815982 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22hwm\" (UniqueName: \"kubernetes.io/projected/ec9a7008-5e58-4c36-b618-c2d808660e9c-kube-api-access-22hwm\") on node \"ip-10-0-128-224.ec2.internal\" DevicePath \"\"" Jul 16 00:55:55.484891 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:55:55.484848 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec9a7008-5e58-4c36-b618-c2d808660e9c" path="/var/lib/kubelet/pods/ec9a7008-5e58-4c36-b618-c2d808660e9c/volumes" Jul 16 00:56:02.331159 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:02.331132 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dd579ccfb-55kdl_d0656155-9214-4f67-bbc0-623164c47b48/copy-manifests/0.log" Jul 16 00:56:02.344321 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:02.344293 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dd579ccfb-55kdl_d0656155-9214-4f67-bbc0-623164c47b48/manager/0.log" Jul 16 00:56:04.459629 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:04.459599 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-derive-0aafd8c9-29736045-jvfsj_de0017b7-e081-426b-b95a-1d1485294665/cleanup/0.log" Jul 16 00:56:04.937406 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:04.937370 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vtmws"] Jul 16 00:56:06.697707 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:06.697675 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-vtmws_4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807/limitador/0.log" Jul 16 00:56:07.184560 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:07.184501 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46kft2_10abc4f8-62a9-47ff-9058-d89bb8de1c94/istio-proxy/0.log" Jul 16 00:56:07.312505 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:07.312464 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vkk98_f79ee54c-673f-4800-9fae-ece9824fb6eb/discovery/0.log" Jul 16 00:56:08.471465 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:08.471439 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qbh4k_de2e3873-bae7-47ae-9c82-8b7e33c39972/main/0.log" Jul 16 00:56:08.808882 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:08.808837 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-skkz5_edf2d7a0-43b8-4501-a145-66c90136c0fd/main/0.log" Jul 16 00:56:17.203081 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:17.203048 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xl7gl_868d72f4-c70a-47ab-842d-5d22e5586970/global-pull-secret-syncer/0.log" Jul 16 00:56:17.282332 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:17.282289 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-btt64_8a49dd8c-7e28-416a-9e61-102f27d5c7df/konnectivity-agent/0.log" Jul 16 00:56:17.334181 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:17.334127 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-224.ec2.internal_e9f64c09e4b0a625629b9a3434bf1892/haproxy/0.log" Jul 16 00:56:22.601946 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:22.601871 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-vtmws_4d7292d3-ce4f-4dcd-bf7d-c171cd9ad807/limitador/0.log" Jul 16 00:56:24.539340 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:24.539306 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-58d9q_4c9734b3-3d18-4827-aaa0-898ac71c02bb/kube-state-metrics/0.log" Jul 16 00:56:24.557823 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:24.557783 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-58d9q_4c9734b3-3d18-4827-aaa0-898ac71c02bb/kube-rbac-proxy-main/0.log" Jul 16 00:56:24.578704 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:24.578676 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-58d9q_4c9734b3-3d18-4827-aaa0-898ac71c02bb/kube-rbac-proxy-self/0.log" Jul 16 00:56:24.741405 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:24.741381 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9zq5h_733af086-2bdf-4a4d-bec0-61bc4e461723/node-exporter/0.log" Jul 16 00:56:24.781821 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:24.781800 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9zq5h_733af086-2bdf-4a4d-bec0-61bc4e461723/kube-rbac-proxy/0.log" Jul 16 00:56:24.807691 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:24.807669 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9zq5h_733af086-2bdf-4a4d-bec0-61bc4e461723/init-textfile/0.log" Jul 16 00:56:24.997195 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:24.997165 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-nbq5b_705ef190-5007-4564-b4f2-8b714fe92eb0/kube-rbac-proxy-main/0.log" Jul 16 00:56:25.029703 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:25.029677 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-nbq5b_705ef190-5007-4564-b4f2-8b714fe92eb0/kube-rbac-proxy-self/0.log" Jul 16 00:56:25.051626 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:25.051553 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-nbq5b_705ef190-5007-4564-b4f2-8b714fe92eb0/openshift-state-metrics/0.log" Jul 16 00:56:25.089655 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:25.089630 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbb20ce4-f1f6-484e-b481-d270387cb6f9/prometheus/0.log" Jul 16 00:56:25.110023 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:25.109998 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbb20ce4-f1f6-484e-b481-d270387cb6f9/config-reloader/0.log" Jul 16 00:56:25.128795 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:25.128775 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbb20ce4-f1f6-484e-b481-d270387cb6f9/thanos-sidecar/0.log" Jul 16 00:56:25.161118 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:25.161090 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbb20ce4-f1f6-484e-b481-d270387cb6f9/kube-rbac-proxy-web/0.log" Jul 16 00:56:25.183235 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:25.183213 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbb20ce4-f1f6-484e-b481-d270387cb6f9/kube-rbac-proxy/0.log" Jul 16 00:56:25.203079 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:25.203054 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbb20ce4-f1f6-484e-b481-d270387cb6f9/kube-rbac-proxy-thanos/0.log" Jul 16 00:56:25.223056 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:25.223030 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbb20ce4-f1f6-484e-b481-d270387cb6f9/init-config-reloader/0.log" Jul 16 00:56:26.672968 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.672938 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268"] Jul 16 00:56:26.673338 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.673259 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac88e9c0-e318-47af-ae14-5341dc95892f" containerName="main" Jul 16 00:56:26.673338 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.673271 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac88e9c0-e318-47af-ae14-5341dc95892f" containerName="main" Jul 16 00:56:26.673338 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.673283 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec9a7008-5e58-4c36-b618-c2d808660e9c" containerName="istio-proxy" Jul 16 00:56:26.673338 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.673288 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec9a7008-5e58-4c36-b618-c2d808660e9c" containerName="istio-proxy" Jul 16 00:56:26.673338 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.673299 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec9a7008-5e58-4c36-b618-c2d808660e9c" containerName="istio-proxy" Jul 16 00:56:26.673338 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.673305 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec9a7008-5e58-4c36-b618-c2d808660e9c" containerName="istio-proxy" Jul 16 00:56:26.673522 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.673361 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec9a7008-5e58-4c36-b618-c2d808660e9c" containerName="istio-proxy" Jul 16 00:56:26.673522 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.673370 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac88e9c0-e318-47af-ae14-5341dc95892f" containerName="main" Jul 16 00:56:26.678485 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.678451 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.681615 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.681589 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lvdqh\"/\"default-dockercfg-zvzsp\"" Jul 16 00:56:26.681615 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.681604 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lvdqh\"/\"kube-root-ca.crt\"" Jul 16 00:56:26.682648 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.682631 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lvdqh\"/\"openshift-service-ca.crt\"" Jul 16 00:56:26.688378 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.688356 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268"] Jul 16 00:56:26.791761 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.791727 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db86e346-4de2-451b-90b9-2f599df3d78c-sys\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.791761 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.791765 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2cmg\" (UniqueName: \"kubernetes.io/projected/db86e346-4de2-451b-90b9-2f599df3d78c-kube-api-access-t2cmg\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.791981 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.791791 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db86e346-4de2-451b-90b9-2f599df3d78c-proc\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.791981 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.791894 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db86e346-4de2-451b-90b9-2f599df3d78c-lib-modules\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.791981 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.791952 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db86e346-4de2-451b-90b9-2f599df3d78c-podres\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.892869 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.892830 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db86e346-4de2-451b-90b9-2f599df3d78c-lib-modules\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.893052 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.892883 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db86e346-4de2-451b-90b9-2f599df3d78c-podres\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.893052 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.892961 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db86e346-4de2-451b-90b9-2f599df3d78c-sys\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.893052 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.892991 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2cmg\" (UniqueName: \"kubernetes.io/projected/db86e346-4de2-451b-90b9-2f599df3d78c-kube-api-access-t2cmg\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.893052 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.893014 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db86e346-4de2-451b-90b9-2f599df3d78c-sys\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.893052 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.892990 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db86e346-4de2-451b-90b9-2f599df3d78c-lib-modules\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.893052 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.893025 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db86e346-4de2-451b-90b9-2f599df3d78c-proc\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.893287 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.893065 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db86e346-4de2-451b-90b9-2f599df3d78c-proc\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.893287 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.893024 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db86e346-4de2-451b-90b9-2f599df3d78c-podres\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.900963 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.900942 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2cmg\" (UniqueName: \"kubernetes.io/projected/db86e346-4de2-451b-90b9-2f599df3d78c-kube-api-access-t2cmg\") pod \"perf-node-gather-daemonset-zj268\" (UID: \"db86e346-4de2-451b-90b9-2f599df3d78c\") " pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:26.990451 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:26.990367 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:27.120001 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:27.119839 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268"] Jul 16 00:56:27.122128 ip-10-0-128-224 kubenswrapper[2564]: W0716 00:56:27.122102 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddb86e346_4de2_451b_90b9_2f599df3d78c.slice/crio-24b8471e568175e2e34f182cf51000105b4188e27713b6d7fe2fda587835ad89 WatchSource:0}: Error finding container 24b8471e568175e2e34f182cf51000105b4188e27713b6d7fe2fda587835ad89: Status 404 returned error can't find the container with id 24b8471e568175e2e34f182cf51000105b4188e27713b6d7fe2fda587835ad89 Jul 16 00:56:27.332253 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:27.332220 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/1.log" Jul 16 00:56:27.337062 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:27.337036 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nsg4s_89e7200c-a92a-4b5b-aead-a8279280af08/console-operator/2.log" Jul 16 00:56:27.845292 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:27.845261 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-6n2bd_1fb1a268-c9cb-4bb2-bcba-a33d1944ce2c/download-server/0.log" Jul 16 00:56:27.897950 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:27.897924 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" event={"ID":"db86e346-4de2-451b-90b9-2f599df3d78c","Type":"ContainerStarted","Data":"0ddca3da973237af2519e9f4d4946a6d6fc6d7955bb1cf4d954958b2cf5b6668"} Jul 16 00:56:27.898128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:27.897968 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" event={"ID":"db86e346-4de2-451b-90b9-2f599df3d78c","Type":"ContainerStarted","Data":"24b8471e568175e2e34f182cf51000105b4188e27713b6d7fe2fda587835ad89"} Jul 16 00:56:27.898128 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:27.898038 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:27.914984 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:27.914920 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" podStartSLOduration=1.9149069669999998 podStartE2EDuration="1.914906967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 00:56:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 00:56:27.913893528 +0000 UTC m=+2596.981834003" watchObservedRunningTime="2026-07-16 00:56:27.914906967 +0000 UTC m=+2596.982847440" Jul 16 00:56:29.408274 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:29.408245 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k8wkl_f5645e23-1659-4fa0-b4f8-7cc6e7f31d96/dns/0.log" Jul 16 00:56:29.428618 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:29.428589 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k8wkl_f5645e23-1659-4fa0-b4f8-7cc6e7f31d96/kube-rbac-proxy/0.log" Jul 16 00:56:29.498868 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:29.498842 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mcr99_d5ea17c6-9c8f-4862-81f4-3b6c3b380f4c/dns-node-resolver/0.log" Jul 16 00:56:30.112919 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:30.112894 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sf9f6_63cc176e-fbac-4a75-877f-db6742663c92/node-ca/0.log" Jul 16 00:56:31.026847 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:31.026817 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46kft2_10abc4f8-62a9-47ff-9058-d89bb8de1c94/istio-proxy/0.log" Jul 16 00:56:31.130762 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:31.130724 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vkk98_f79ee54c-673f-4800-9fae-ece9824fb6eb/discovery/0.log" Jul 16 00:56:32.023119 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:32.023087 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m5p7c_ce5278f9-b320-4971-92ef-a33f0804bb39/serve-healthcheck-canary/0.log" Jul 16 00:56:32.619088 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:32.619059 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m9xxt_acb05a14-8ed0-4a7c-be13-e995fb05ebc6/kube-rbac-proxy/0.log" Jul 16 00:56:32.638826 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:32.638802 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m9xxt_acb05a14-8ed0-4a7c-be13-e995fb05ebc6/exporter/0.log" Jul 16 00:56:32.661004 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:32.660978 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m9xxt_acb05a14-8ed0-4a7c-be13-e995fb05ebc6/extractor/0.log" Jul 16 00:56:33.912644 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:33.912616 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lvdqh/perf-node-gather-daemonset-zj268" Jul 16 00:56:35.157973 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:35.157938 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dd579ccfb-55kdl_d0656155-9214-4f67-bbc0-623164c47b48/manager/0.log" Jul 16 00:56:35.176939 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:35.176912 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dd579ccfb-55kdl_d0656155-9214-4f67-bbc0-623164c47b48/copy-manifests/0.log" Jul 16 00:56:36.396971 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:36.396934 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58f9b8cc9c-g86sd_9837e6d6-bc15-45c4-b3ba-b8f102cac92b/manager/0.log" Jul 16 00:56:36.442646 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:36.442616 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-wwkf4_186d486a-064b-4297-92a4-eebf3c252208/openshift-lws-operator/0.log" Jul 16 00:56:36.646008 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:36.645933 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46kft2_10abc4f8-62a9-47ff-9058-d89bb8de1c94/istio-proxy/0.log" Jul 16 00:56:36.745549 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:36.745503 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vkk98_f79ee54c-673f-4800-9fae-ece9824fb6eb/discovery/0.log" Jul 16 00:56:38.190312 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:38.190286 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46kft2_10abc4f8-62a9-47ff-9058-d89bb8de1c94/istio-proxy/0.log" Jul 16 00:56:38.280499 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:38.280465 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vkk98_f79ee54c-673f-4800-9fae-ece9824fb6eb/discovery/0.log" Jul 16 00:56:41.801883 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:41.801845 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-bm2b9_6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5/kube-storage-version-migrator-operator/1.log" Jul 16 00:56:41.802749 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:41.802732 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-bm2b9_6b2fa53b-3900-4a5c-bf5f-f5a4cd8a35b5/kube-storage-version-migrator-operator/0.log" Jul 16 00:56:42.897057 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:42.897030 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rltww_4089f83d-6157-48c3-8ed0-aa5bf7f01f7b/kube-multus-additional-cni-plugins/0.log" Jul 16 00:56:42.917219 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:42.917195 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rltww_4089f83d-6157-48c3-8ed0-aa5bf7f01f7b/egress-router-binary-copy/0.log" Jul 16 00:56:42.936127 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:42.936103 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rltww_4089f83d-6157-48c3-8ed0-aa5bf7f01f7b/cni-plugins/0.log" Jul 16 00:56:42.956463 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:42.956442 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rltww_4089f83d-6157-48c3-8ed0-aa5bf7f01f7b/bond-cni-plugin/0.log" Jul 16 00:56:42.975776 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:42.975753 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rltww_4089f83d-6157-48c3-8ed0-aa5bf7f01f7b/routeoverride-cni/0.log" Jul 16 00:56:42.995390 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:42.995367 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rltww_4089f83d-6157-48c3-8ed0-aa5bf7f01f7b/whereabouts-cni-bincopy/0.log" Jul 16 00:56:43.014882 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:43.014862 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rltww_4089f83d-6157-48c3-8ed0-aa5bf7f01f7b/whereabouts-cni/0.log" Jul 16 00:56:43.253516 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:43.253437 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-r5xhf_6487ac13-1c94-4d7f-917b-f2f14892e58d/kube-multus/0.log" Jul 16 00:56:43.320343 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:43.320308 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l725h_5f9ca78d-1eb6-45bc-bf9a-01b646191c67/network-metrics-daemon/0.log" Jul 16 00:56:43.337949 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:43.337923 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l725h_5f9ca78d-1eb6-45bc-bf9a-01b646191c67/kube-rbac-proxy/0.log" Jul 16 00:56:44.150969 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:44.150939 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-controller/0.log" Jul 16 00:56:44.170194 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:44.170167 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/0.log" Jul 16 00:56:44.170194 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:44.170188 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovn-acl-logging/1.log" Jul 16 00:56:44.193694 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:44.193665 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/kube-rbac-proxy-node/0.log" Jul 16 00:56:44.218756 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:44.218727 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/kube-rbac-proxy-ovn-metrics/0.log" Jul 16 00:56:44.239269 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:44.239248 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/northd/0.log" Jul 16 00:56:44.259552 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:44.259513 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/nbdb/0.log" Jul 16 00:56:44.280999 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:44.280976 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/sbdb/0.log" Jul 16 00:56:44.325018 ip-10-0-128-224 kubenswrapper[2564]: I0716 00:56:44.324992 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qrwnm_827e2908-fba6-44a1-bfb4-6b22753d0eee/ovnkube-controller/0.log"