Jul 20 04:57:25.244285 ip-10-0-128-33 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 20 04:57:25.244302 ip-10-0-128-33 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 20 04:57:25.244311 ip-10-0-128-33 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 20 04:57:25.244681 ip-10-0-128-33 systemd[1]: Failed to start Kubernetes Kubelet. Jul 20 04:57:34.881006 ip-10-0-128-33 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 20 04:57:34.881020 ip-10-0-128-33 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e5d5aa2b5031468893e58d56535b5838 -- Jul 20 04:59:16.860288 ip-10-0-128-33 systemd[1]: Starting Kubernetes Kubelet... Jul 20 04:59:17.476798 ip-10-0-128-33 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 04:59:17.476798 ip-10-0-128-33 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 20 04:59:17.476798 ip-10-0-128-33 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 04:59:17.476798 ip-10-0-128-33 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 20 04:59:17.476798 ip-10-0-128-33 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 04:59:17.478425 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.478339 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 20 04:59:17.482176 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482151 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 04:59:17.482176 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482177 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 04:59:17.482176 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482182 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 04:59:17.482176 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482185 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482188 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482191 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482195 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482197 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482202 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482206 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482209 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482212 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482215 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482218 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482221 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482224 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482226 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482229 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482232 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482235 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482238 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482241 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 04:59:17.482284 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482243 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482246 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482249 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482251 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482253 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482256 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482259 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482261 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482264 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482266 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482269 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482278 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482281 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482283 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482286 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482289 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482293 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482296 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482300 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482302 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 04:59:17.482712 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482305 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482307 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482310 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482313 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482315 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482318 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482320 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482323 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482325 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482327 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482330 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482333 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482335 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482338 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482341 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482344 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482347 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482349 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482352 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482354 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 04:59:17.483227 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482357 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482359 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482362 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482365 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482369 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482372 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482375 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482379 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482382 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482385 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482388 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482390 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482393 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482395 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482398 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482401 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482403 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482406 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482408 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482411 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 04:59:17.483691 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482413 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482416 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482418 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482421 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482423 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482426 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482428 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482431 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482434 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482436 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482441 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482444 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482447 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482449 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482452 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482455 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.482457 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483154 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483162 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 04:59:17.484175 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483166 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483168 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483171 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483175 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483178 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483181 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483183 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483186 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483189 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483191 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483194 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483197 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483200 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483202 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483207 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483210 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483214 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483216 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483219 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 04:59:17.484665 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483223 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483226 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483230 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483234 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483237 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483239 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483242 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483245 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483247 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483250 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483253 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483255 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483258 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483261 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483263 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483266 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483268 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483271 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483274 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483277 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 04:59:17.485163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483280 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483282 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483285 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483287 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483290 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483292 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483295 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483298 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483301 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483303 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483305 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483308 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483311 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483314 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483317 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483319 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483322 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483324 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483327 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483329 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 04:59:17.485635 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483332 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483335 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483337 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483340 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483343 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483345 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483348 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483351 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483353 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483356 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483359 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483361 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483364 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483366 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483369 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483371 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483374 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483376 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483379 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483381 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483384 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 04:59:17.486115 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483386 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483389 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483391 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483394 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483398 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483400 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483403 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483405 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483408 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483410 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483413 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483415 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483418 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483420 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483425 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483428 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483431 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483798 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483803 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 04:59:17.486660 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483806 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483809 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483813 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483816 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483818 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483821 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483824 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483827 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483830 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483832 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483835 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483837 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483840 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483842 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483845 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483847 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483851 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483854 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483856 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483859 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 04:59:17.487148 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483861 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483864 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483867 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483869 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483872 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483877 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483880 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483882 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483884 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483887 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483889 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483892 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483894 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483898 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483902 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483905 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483909 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483912 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483915 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 04:59:17.487630 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483918 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483920 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483923 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483926 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483930 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483934 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483936 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483939 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483942 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483945 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483947 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483950 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483967 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483971 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483975 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483979 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483983 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483986 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483990 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483993 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 04:59:17.488078 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483996 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.483998 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484001 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484004 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484006 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484009 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484011 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484014 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484016 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484020 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484023 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484026 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484028 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484032 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484035 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484037 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484040 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484042 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484045 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 04:59:17.488546 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484047 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484050 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484052 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484055 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484058 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484061 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484063 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484066 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484068 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484071 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484073 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484077 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484080 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484082 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484085 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484088 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484090 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484093 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484095 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484164 2576 flags.go:64] FLAG: --address="0.0.0.0" Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484171 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 20 04:59:17.489065 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484177 2576 flags.go:64] FLAG: --anonymous-auth="true" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484182 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484187 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484190 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484194 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484200 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484204 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484207 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484210 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484213 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484217 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484220 2576 flags.go:64] FLAG: --cgroup-root="" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484222 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484225 2576 flags.go:64] FLAG: --client-ca-file="" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484228 2576 flags.go:64] FLAG: --cloud-provider="external" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484231 2576 flags.go:64] FLAG: --cluster-dns="[]" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484236 2576 flags.go:64] FLAG: --cluster-domain="" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484239 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484242 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484245 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484248 2576 flags.go:64] FLAG: --container-log-max-files="5" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484252 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484256 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 20 04:59:17.489554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484259 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484262 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484265 2576 flags.go:64] FLAG: --contention-profiling="false" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484268 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484271 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484274 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484277 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484289 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484292 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484295 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484298 2576 flags.go:64] FLAG: --enable-load-reader="false" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484301 2576 flags.go:64] FLAG: --enable-server="true" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484305 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484309 2576 flags.go:64] FLAG: --event-burst="100" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484316 2576 flags.go:64] FLAG: --event-qps="50" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484319 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484322 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484325 2576 flags.go:64] FLAG: --eviction-hard="" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484328 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484331 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484334 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484337 2576 flags.go:64] FLAG: --eviction-soft="" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484340 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484343 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484346 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 20 04:59:17.490103 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484349 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484352 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484354 2576 flags.go:64] FLAG: --fail-swap-on="true" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484357 2576 flags.go:64] FLAG: --feature-gates="" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484361 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484364 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484368 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484371 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484374 2576 flags.go:64] FLAG: --healthz-port="10248" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484377 2576 flags.go:64] FLAG: --help="false" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484380 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484383 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484387 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484390 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484393 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484396 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484399 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484402 2576 flags.go:64] FLAG: --image-service-endpoint="" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484405 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484408 2576 flags.go:64] FLAG: --kube-api-burst="100" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484411 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484416 2576 flags.go:64] FLAG: --kube-api-qps="50" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484419 2576 flags.go:64] FLAG: --kube-reserved="" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484422 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 20 04:59:17.490736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484424 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484428 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484431 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484433 2576 flags.go:64] FLAG: --lock-file="" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484436 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484439 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484442 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484447 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484450 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484453 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484456 2576 flags.go:64] FLAG: --logging-format="text" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484458 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484462 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484465 2576 flags.go:64] FLAG: --manifest-url="" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484468 2576 flags.go:64] FLAG: --manifest-url-header="" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484472 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484475 2576 flags.go:64] FLAG: --max-open-files="1000000" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484479 2576 flags.go:64] FLAG: --max-pods="110" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484482 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484485 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484488 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484491 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484494 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484497 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484500 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 20 04:59:17.491344 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484507 2576 flags.go:64] FLAG: --node-status-max-images="50" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484510 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484513 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484516 2576 flags.go:64] FLAG: --pod-cidr="" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484521 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484526 2576 flags.go:64] FLAG: --pod-manifest-path="" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484529 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484533 2576 flags.go:64] FLAG: --pods-per-core="0" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484536 2576 flags.go:64] FLAG: --port="10250" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484539 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484542 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-062ddad64a93cac7e" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484545 2576 flags.go:64] FLAG: --qos-reserved="" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484548 2576 flags.go:64] FLAG: --read-only-port="10255" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484551 2576 flags.go:64] FLAG: --register-node="true" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484554 2576 flags.go:64] FLAG: --register-with-taints="" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484557 2576 flags.go:64] FLAG: --registry-burst="10" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484560 2576 flags.go:64] FLAG: --registry-qps="5" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484563 2576 flags.go:64] FLAG: --reserved-cpus="" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484566 2576 flags.go:64] FLAG: --reserved-memory="" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484569 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484573 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484575 2576 flags.go:64] FLAG: --rotate-certificates="false" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484578 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484581 2576 flags.go:64] FLAG: --runonce="false" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484584 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 20 04:59:17.491947 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484587 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484590 2576 flags.go:64] FLAG: --seccomp-default="false" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484593 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484596 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484599 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484603 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484605 2576 flags.go:64] FLAG: --storage-driver-password="root" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484608 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484611 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484614 2576 flags.go:64] FLAG: --storage-driver-user="root" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484617 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484620 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484625 2576 flags.go:64] FLAG: --system-cgroups="" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484627 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484633 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484635 2576 flags.go:64] FLAG: --tls-cert-file="" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484638 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484643 2576 flags.go:64] FLAG: --tls-min-version="" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484646 2576 flags.go:64] FLAG: --tls-private-key-file="" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484648 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484651 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484654 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484657 2576 flags.go:64] FLAG: --v="2" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484661 2576 flags.go:64] FLAG: --version="false" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484665 2576 flags.go:64] FLAG: --vmodule="" Jul 20 04:59:17.492568 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484669 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.484675 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484792 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484797 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484800 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484802 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484805 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484808 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484810 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484813 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484816 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484819 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484821 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484823 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484826 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484828 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484831 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484834 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484836 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484840 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484843 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 04:59:17.493163 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484846 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484849 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484851 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484854 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484856 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484859 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484861 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484864 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484866 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484869 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484871 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484874 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484878 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484881 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484883 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484886 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484888 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484891 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484893 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484896 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 04:59:17.493663 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484898 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484901 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484903 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484906 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484909 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484911 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484914 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484916 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484919 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484921 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484925 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484928 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484931 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484935 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484939 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484942 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484944 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484947 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484949 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 04:59:17.494198 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484964 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484967 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484970 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484973 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484975 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484980 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484982 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484985 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484988 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484990 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484993 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484996 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.484998 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485001 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485003 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485006 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485008 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485011 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485013 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485016 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 04:59:17.494652 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485018 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485021 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485023 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485027 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485030 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485032 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485035 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485038 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485042 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485046 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485049 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485051 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485054 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485057 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485060 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485064 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485067 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485071 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485074 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 04:59:17.495140 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485076 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.485079 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.485677 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.492648 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.492664 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.493260 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.493268 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493334 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493340 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493343 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493347 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493351 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493355 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493358 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493361 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 04:59:17.495594 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493364 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493367 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493370 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493372 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493375 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493378 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493380 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493383 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493385 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493388 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493391 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493393 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493396 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493398 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493401 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493404 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493406 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493409 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493411 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493414 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 04:59:17.495987 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493416 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493426 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493429 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493432 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493434 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493437 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493439 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493442 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493444 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493447 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493449 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493452 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493455 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493457 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493460 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493462 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493465 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493467 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493470 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493472 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 04:59:17.496492 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493475 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493477 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493480 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493482 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493485 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493488 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493490 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493493 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493495 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493498 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493500 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493503 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493505 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493508 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493517 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493520 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493522 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493525 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493529 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 04:59:17.496977 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493533 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493535 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493538 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493540 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493543 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493546 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493550 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493553 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493556 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493559 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493562 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493564 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493567 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493569 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493572 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493574 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493577 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493579 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493582 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 04:59:17.497439 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493585 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493587 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493590 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493592 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493595 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493597 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493600 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493603 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493605 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493614 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493616 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493619 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493622 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.493627 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493797 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 04:59:17.497884 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493802 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493805 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493808 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493811 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493813 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493816 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493824 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493827 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493829 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493832 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493834 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493837 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493839 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493842 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493844 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493848 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493852 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493855 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493858 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 04:59:17.498270 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493860 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493863 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493866 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493868 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493871 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493874 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493877 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493879 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493889 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493893 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493896 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493898 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493901 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493904 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493906 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493909 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493912 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493914 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493917 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 04:59:17.498714 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493920 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493922 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493925 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493927 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493929 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493932 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493935 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493937 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493939 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493942 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493944 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493947 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493949 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493966 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493969 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493972 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493976 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493979 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493982 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493986 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 04:59:17.499282 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493989 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.493991 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494000 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494003 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494005 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494008 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494010 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494013 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494015 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494018 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494020 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494022 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494025 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494027 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494030 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494032 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494035 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494037 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494039 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494042 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 04:59:17.499793 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494044 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494047 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494050 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494052 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494054 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494058 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494060 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494062 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494065 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494068 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494070 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494073 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494076 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494078 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494081 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494089 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494091 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494094 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494096 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 04:59:17.500293 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:17.494099 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 04:59:17.500743 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.494103 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 04:59:17.500743 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.494221 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jul 20 04:59:17.500743 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.496328 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 20 04:59:17.500743 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.497464 2576 server.go:1019] "Starting client certificate rotation" Jul 20 04:59:17.500743 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.497568 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 04:59:17.500743 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.498169 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 04:59:17.500743 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.499660 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 04:59:17.505167 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.505149 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 04:59:17.513097 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.513079 2576 log.go:25] "Validated CRI v1 runtime API" Jul 20 04:59:17.514971 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.514941 2576 log.go:25] "Validated CRI v1 image API" Jul 20 04:59:17.515859 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.515841 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 20 04:59:17.517794 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.517774 2576 fs.go:135] Filesystem UUIDs: map[16556570-1f45-4642-b5ad-a3ca980086d1:/dev/nvme0n1p3 22a1b805-262f-4223-a809-cd08a6327b0c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 20 04:59:17.517855 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.517795 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 20 04:59:17.523371 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.523266 2576 manager.go:216] Machine: {Timestamp:2026-07-20 04:59:17.522143569 +0000 UTC m=+0.568670616 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101878 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec258a6e5f387243593368b7df38b5ce SystemUUID:ec258a6e-5f38-7243-5933-68b7df38b5ce BootID:e5d5aa2b-5031-4688-93e5-8d56535b5838 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:58:cc:f3:f9:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:58:cc:f3:f9:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:60:08:c2:f7:d2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 20 04:59:17.523371 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.523365 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 20 04:59:17.523487 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.523452 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 20 04:59:17.525834 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.525808 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 20 04:59:17.526008 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.525836 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-33.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 20 04:59:17.526876 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.526865 2576 topology_manager.go:138] "Creating topology manager with none policy" Jul 20 04:59:17.526923 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.526878 2576 container_manager_linux.go:309] "Creating device plugin manager" Jul 20 04:59:17.526923 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.526886 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 04:59:17.527369 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.527359 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 04:59:17.528416 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.528405 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 20 04:59:17.529006 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.528990 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 04:59:17.532047 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.532034 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 20 04:59:17.532159 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.532150 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 20 04:59:17.532199 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.532180 2576 kubelet.go:486] "Attempting to sync node with API server" Jul 20 04:59:17.532199 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.532188 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 20 04:59:17.532256 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.532232 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 20 04:59:17.532256 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.532241 2576 kubelet.go:398] "Adding apiserver pod source" Jul 20 04:59:17.532256 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.532250 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 20 04:59:17.532652 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.532640 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 04:59:17.532683 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.532660 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 04:59:17.534880 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.534864 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 20 04:59:17.536257 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.536241 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 20 04:59:17.536327 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.536273 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 20 04:59:17.537226 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.537215 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 20 04:59:17.537262 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.537232 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 20 04:59:17.537262 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.537239 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 20 04:59:17.537262 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.537244 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 20 04:59:17.537262 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.537253 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 20 04:59:17.537262 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.537261 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 20 04:59:17.537433 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.537272 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 20 04:59:17.537433 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.537280 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 20 04:59:17.537433 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.537287 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 20 04:59:17.537433 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.537293 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 20 04:59:17.537433 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.537326 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 20 04:59:17.537433 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.537333 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 20 04:59:17.538528 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.538515 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 20 04:59:17.538528 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.538525 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 20 04:59:17.541741 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.541726 2576 server.go:1268] "Started kubelet" Jul 20 04:59:17.541922 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.541838 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 20 04:59:17.542376 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.541814 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 20 04:59:17.542474 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.542398 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 20 04:59:17.542915 ip-10-0-128-33 systemd[1]: Started Kubernetes Kubelet. Jul 20 04:59:17.543146 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.543018 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-33.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 20 04:59:17.543813 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.543356 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 20 04:59:17.543813 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.543219 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-33.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 20 04:59:17.545918 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.545898 2576 server.go:310] "Adding debug handlers to kubelet server" Jul 20 04:59:17.546928 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.546913 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 20 04:59:17.547977 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.547938 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 20 04:59:17.549002 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.548986 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gvh2s" Jul 20 04:59:17.550460 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.549772 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-33.ec2.internal.18c3e6b88496cd80 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-33.ec2.internal,UID:ip-10-0-128-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-33.ec2.internal,},FirstTimestamp:2026-07-20 04:59:17.541694848 +0000 UTC m=+0.588221744,LastTimestamp:2026-07-20 04:59:17.541694848 +0000 UTC m=+0.588221744,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-33.ec2.internal,}" Jul 20 04:59:17.552175 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.552157 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 20 04:59:17.552175 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.552168 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 20 04:59:17.552296 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.552258 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:17.552779 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.552729 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 20 04:59:17.552855 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.552797 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 20 04:59:17.552855 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.552751 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 20 04:59:17.553009 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.552994 2576 reconstruct.go:97] "Volume reconstruction finished" Jul 20 04:59:17.553009 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.553008 2576 reconciler.go:29] "Reconciler: start to sync state" Jul 20 04:59:17.553111 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.553019 2576 factory.go:55] Registering systemd factory Jul 20 04:59:17.553111 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.553037 2576 factory.go:223] Registration of the systemd container factory successfully Jul 20 04:59:17.553265 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.553251 2576 factory.go:153] Registering CRI-O factory Jul 20 04:59:17.553313 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.553271 2576 factory.go:223] Registration of the crio container factory successfully Jul 20 04:59:17.553364 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.553316 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 20 04:59:17.553364 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.553339 2576 factory.go:103] Registering Raw factory Jul 20 04:59:17.553364 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.553352 2576 manager.go:1196] Started watching for new ooms in manager Jul 20 04:59:17.553775 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.553762 2576 manager.go:349] Starting recovery of all containers Jul 20 04:59:17.558387 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.558365 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gvh2s" Jul 20 04:59:17.559120 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.559095 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 20 04:59:17.559209 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.559149 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-33.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 20 04:59:17.564171 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.564156 2576 manager.go:354] Recovery completed Jul 20 04:59:17.570216 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.570202 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 04:59:17.571913 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.571895 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientMemory" Jul 20 04:59:17.572005 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.571927 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 04:59:17.572005 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.571939 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientPID" Jul 20 04:59:17.572612 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.572597 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 20 04:59:17.572612 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.572610 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 20 04:59:17.572754 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.572625 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 20 04:59:17.572754 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.572604 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 04:59:17.573479 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.573462 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientMemory" Jul 20 04:59:17.573545 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.573494 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 04:59:17.573545 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.573508 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientPID" Jul 20 04:59:17.573807 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.573745 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-33.ec2.internal.18c3e6b88663e13f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-33.ec2.internal,UID:ip-10-0-128-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-33.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-33.ec2.internal,},FirstTimestamp:2026-07-20 04:59:17.571911999 +0000 UTC m=+0.618438896,LastTimestamp:2026-07-20 04:59:17.571911999 +0000 UTC m=+0.618438896,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-33.ec2.internal,}" Jul 20 04:59:17.574698 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.574687 2576 policy_none.go:49] "None policy: Start" Jul 20 04:59:17.574736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.574702 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 20 04:59:17.574736 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.574712 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 20 04:59:17.576714 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.576702 2576 policy_none.go:47] "Start" Jul 20 04:59:17.622738 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.622559 2576 manager.go:337] "Starting Device Plugin manager" Jul 20 04:59:17.622851 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.622765 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 20 04:59:17.622851 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.622779 2576 server.go:87] "Starting device plugin registration server" Jul 20 04:59:17.623085 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.623072 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 20 04:59:17.623167 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.623089 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 20 04:59:17.623214 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.623200 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 20 04:59:17.623308 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.623292 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 20 04:59:17.623308 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.623304 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 20 04:59:17.623870 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.623848 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 20 04:59:17.623938 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.623916 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:17.645443 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.645415 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 20 04:59:17.654231 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.646518 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 20 04:59:17.654231 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.646554 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 20 04:59:17.654231 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.646576 2576 kubelet.go:2448] "Starting kubelet main sync loop" Jul 20 04:59:17.654231 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.646608 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 20 04:59:17.654231 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.648945 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 04:59:17.725067 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.725029 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 04:59:17.727100 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.727071 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientMemory" Jul 20 04:59:17.727100 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.727104 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 04:59:17.727257 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.727120 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientPID" Jul 20 04:59:17.727257 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.727147 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.733956 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.733939 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.734016 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.733966 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-33.ec2.internal\": node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:17.747260 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.747234 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-33.ec2.internal"] Jul 20 04:59:17.747350 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.747297 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 04:59:17.748186 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.748167 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientMemory" Jul 20 04:59:17.748294 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.748199 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 04:59:17.748294 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.748209 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientPID" Jul 20 04:59:17.749397 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.749375 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:17.750777 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.750763 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 04:59:17.750900 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.750887 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.750951 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.750933 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 04:59:17.751772 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.751756 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientMemory" Jul 20 04:59:17.751772 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.751768 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientMemory" Jul 20 04:59:17.751930 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.751785 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 04:59:17.751930 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.751792 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 04:59:17.751930 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.751800 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientPID" Jul 20 04:59:17.751930 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.751803 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientPID" Jul 20 04:59:17.754356 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.754340 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.754424 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.754367 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 04:59:17.755226 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.755208 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientMemory" Jul 20 04:59:17.755307 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.755237 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 04:59:17.755307 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.755248 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeHasSufficientPID" Jul 20 04:59:17.768631 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.768613 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-33.ec2.internal\" not found" node="ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.772771 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.772754 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-33.ec2.internal\" not found" node="ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.850135 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.850107 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:17.853565 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.853511 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c80a066a6bda5fad4b7c26559d43b6cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal\" (UID: \"c80a066a6bda5fad4b7c26559d43b6cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.853565 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.853543 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8c6bf12d5d282ff52e7d38919e9afae-config\") pod \"kube-apiserver-proxy-ip-10-0-128-33.ec2.internal\" (UID: \"a8c6bf12d5d282ff52e7d38919e9afae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.853565 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.853564 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c80a066a6bda5fad4b7c26559d43b6cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal\" (UID: \"c80a066a6bda5fad4b7c26559d43b6cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.951182 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:17.951154 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:17.954706 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.954685 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c80a066a6bda5fad4b7c26559d43b6cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal\" (UID: \"c80a066a6bda5fad4b7c26559d43b6cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.954754 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.954716 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c80a066a6bda5fad4b7c26559d43b6cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal\" (UID: \"c80a066a6bda5fad4b7c26559d43b6cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.954754 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.954733 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8c6bf12d5d282ff52e7d38919e9afae-config\") pod \"kube-apiserver-proxy-ip-10-0-128-33.ec2.internal\" (UID: \"a8c6bf12d5d282ff52e7d38919e9afae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.954830 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.954784 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8c6bf12d5d282ff52e7d38919e9afae-config\") pod \"kube-apiserver-proxy-ip-10-0-128-33.ec2.internal\" (UID: \"a8c6bf12d5d282ff52e7d38919e9afae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.954830 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.954792 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c80a066a6bda5fad4b7c26559d43b6cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal\" (UID: \"c80a066a6bda5fad4b7c26559d43b6cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal" Jul 20 04:59:17.954830 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:17.954808 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c80a066a6bda5fad4b7c26559d43b6cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal\" (UID: \"c80a066a6bda5fad4b7c26559d43b6cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal" Jul 20 04:59:18.052543 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:18.052481 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:18.071217 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:18.071186 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal" Jul 20 04:59:18.076644 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:18.076548 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-33.ec2.internal" Jul 20 04:59:18.153685 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:18.153599 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:18.260112 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:18.260082 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:18.361053 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:18.361018 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:18.461952 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:18.461864 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:18.497236 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:18.497203 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 20 04:59:18.497689 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:18.497360 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 04:59:18.562142 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:18.562111 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:18.564412 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:18.564371 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 04:54:18 +0000 UTC" deadline="2028-01-03 02:16:30.59979744 +0000 UTC" Jul 20 04:59:18.564549 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:18.564414 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12765h17m12.035391869s" Jul 20 04:59:18.657748 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:18.657709 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc80a066a6bda5fad4b7c26559d43b6cf.slice/crio-678f2688ab63e378e1ed3497ec240cc4a8cbb33b6f32fc9a5eb54a342616ba1c WatchSource:0}: Error finding container 678f2688ab63e378e1ed3497ec240cc4a8cbb33b6f32fc9a5eb54a342616ba1c: Status 404 returned error can't find the container with id 678f2688ab63e378e1ed3497ec240cc4a8cbb33b6f32fc9a5eb54a342616ba1c Jul 20 04:59:18.659011 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:18.658993 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c6bf12d5d282ff52e7d38919e9afae.slice/crio-73d9ebab32d3319233d9f49a4bb9f96b887789a58802e8e0078295665c115f89 WatchSource:0}: Error finding container 73d9ebab32d3319233d9f49a4bb9f96b887789a58802e8e0078295665c115f89: Status 404 returned error can't find the container with id 73d9ebab32d3319233d9f49a4bb9f96b887789a58802e8e0078295665c115f89 Jul 20 04:59:18.662247 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:18.662224 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:18.662832 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:18.662817 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 04:59:18.762903 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:18.762789 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:18.863147 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:18.863095 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-33.ec2.internal\" not found" Jul 20 04:59:18.945857 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:18.945815 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 04:59:18.953345 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:18.953325 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal" Jul 20 04:59:18.964660 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:18.964635 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 04:59:18.968507 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:18.968477 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-33.ec2.internal" Jul 20 04:59:18.978007 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:18.977985 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 04:59:19.037091 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.037012 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 04:59:19.102693 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.102663 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 04:59:19.533648 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.533565 2576 apiserver.go:52] "Watching apiserver" Jul 20 04:59:19.541803 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.541775 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 20 04:59:19.542314 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.542281 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-dh2cl","kube-system/konnectivity-agent-4ljhn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj","openshift-dns/node-resolver-js7gh","openshift-image-registry/node-ca-p29tq","openshift-multus/multus-lpkbr","openshift-multus/network-metrics-daemon-gnmjm","openshift-network-diagnostics/network-check-target-nwhn4","kube-system/kube-apiserver-proxy-ip-10-0-128-33.ec2.internal","openshift-cluster-node-tuning-operator/tuned-lffh5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal","openshift-multus/multus-additional-cni-plugins-lhssz","openshift-network-operator/iptables-alerter-2zdwn","openshift-ovn-kubernetes/ovnkube-node-85lxq"] Jul 20 04:59:19.545139 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.545113 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:19.545287 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:19.545209 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:19.548114 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.548024 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2zdwn" Jul 20 04:59:19.550556 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.550534 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.550877 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.550686 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 20 04:59:19.551297 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.551085 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 20 04:59:19.551297 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.551139 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 20 04:59:19.551297 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.551277 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z4fdg\"" Jul 20 04:59:19.552696 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.552675 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 20 04:59:19.552797 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.552759 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-js7gh" Jul 20 04:59:19.553561 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.553540 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 20 04:59:19.553651 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.553562 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 20 04:59:19.553837 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.553815 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ns7pw\"" Jul 20 04:59:19.553937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.553868 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 20 04:59:19.555309 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.555284 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 20 04:59:19.555552 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.555532 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p29tq" Jul 20 04:59:19.555676 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.555652 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 20 04:59:19.555988 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.555915 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7j556\"" Jul 20 04:59:19.558018 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.557999 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.558132 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.558043 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 20 04:59:19.558527 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.558485 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8bzk2\"" Jul 20 04:59:19.558624 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.558595 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 20 04:59:19.558811 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.558711 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 20 04:59:19.562194 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.561399 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.562194 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.561607 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 20 04:59:19.562194 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.561836 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-64gwh\"" Jul 20 04:59:19.562504 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.562484 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 20 04:59:19.562589 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.562505 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 20 04:59:19.562589 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.562576 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 20 04:59:19.564010 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.563991 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:19.564127 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:19.564083 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:19.564298 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.564272 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 20 04:59:19.564386 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.564361 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 20 04:59:19.564504 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.564469 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jqzrx\"" Jul 20 04:59:19.564694 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.564631 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 20 04:59:19.564798 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.564730 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 20 04:59:19.565732 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.565708 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 04:59:19.565841 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.565733 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 20 04:59:19.565841 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.565720 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 20 04:59:19.568343 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.568024 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4ljhn" Jul 20 04:59:19.569165 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569019 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-device-dir\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.569165 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569081 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-sys-fs\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.569165 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569111 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brzdv\" (UniqueName: \"kubernetes.io/projected/f7fe0af3-add4-47d8-81db-87b049070630-kube-api-access-brzdv\") pod \"iptables-alerter-2zdwn\" (UID: \"f7fe0af3-add4-47d8-81db-87b049070630\") " pod="openshift-network-operator/iptables-alerter-2zdwn" Jul 20 04:59:19.569165 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569141 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xbpw\" (UniqueName: \"kubernetes.io/projected/1edae688-9ce1-49e7-9a37-5d782b643b36-kube-api-access-7xbpw\") pod \"node-resolver-js7gh\" (UID: \"1edae688-9ce1-49e7-9a37-5d782b643b36\") " pod="openshift-dns/node-resolver-js7gh" Jul 20 04:59:19.569440 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569181 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/42007563-e764-4689-a28f-872b48cd1e8b-serviceca\") pod \"node-ca-p29tq\" (UID: \"42007563-e764-4689-a28f-872b48cd1e8b\") " pod="openshift-image-registry/node-ca-p29tq" Jul 20 04:59:19.569440 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569226 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w5kv\" (UniqueName: \"kubernetes.io/projected/42007563-e764-4689-a28f-872b48cd1e8b-kube-api-access-7w5kv\") pod \"node-ca-p29tq\" (UID: \"42007563-e764-4689-a28f-872b48cd1e8b\") " pod="openshift-image-registry/node-ca-p29tq" Jul 20 04:59:19.569440 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569258 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:19.569440 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569284 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.569699 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569674 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1edae688-9ce1-49e7-9a37-5d782b643b36-hosts-file\") pod \"node-resolver-js7gh\" (UID: \"1edae688-9ce1-49e7-9a37-5d782b643b36\") " pod="openshift-dns/node-resolver-js7gh" Jul 20 04:59:19.569802 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569722 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42007563-e764-4689-a28f-872b48cd1e8b-host\") pod \"node-ca-p29tq\" (UID: \"42007563-e764-4689-a28f-872b48cd1e8b\") " pod="openshift-image-registry/node-ca-p29tq" Jul 20 04:59:19.569802 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569748 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.569802 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569780 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7fe0af3-add4-47d8-81db-87b049070630-host-slash\") pod \"iptables-alerter-2zdwn\" (UID: \"f7fe0af3-add4-47d8-81db-87b049070630\") " pod="openshift-network-operator/iptables-alerter-2zdwn" Jul 20 04:59:19.569971 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569831 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rj7z\" (UniqueName: \"kubernetes.io/projected/06a59c1d-4c72-423f-9e8d-27db0a919dba-kube-api-access-9rj7z\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:19.569971 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569876 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7fe0af3-add4-47d8-81db-87b049070630-iptables-alerter-script\") pod \"iptables-alerter-2zdwn\" (UID: \"f7fe0af3-add4-47d8-81db-87b049070630\") " pod="openshift-network-operator/iptables-alerter-2zdwn" Jul 20 04:59:19.569971 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569904 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-registration-dir\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.569971 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.569930 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7x4f\" (UniqueName: \"kubernetes.io/projected/6a61c804-fec7-4b33-96d5-630b09a1eb30-kube-api-access-n7x4f\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.570215 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.570035 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1edae688-9ce1-49e7-9a37-5d782b643b36-tmp-dir\") pod \"node-resolver-js7gh\" (UID: \"1edae688-9ce1-49e7-9a37-5d782b643b36\") " pod="openshift-dns/node-resolver-js7gh" Jul 20 04:59:19.570215 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.570065 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-socket-dir\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.570351 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.570278 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.570351 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.570327 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mxj6s\"" Jul 20 04:59:19.570489 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.570468 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 20 04:59:19.570793 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.570742 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 20 04:59:19.573008 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.572982 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g5vp8\"" Jul 20 04:59:19.573317 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.573296 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 20 04:59:19.573555 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.573536 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 20 04:59:19.574147 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.574128 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:19.574271 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:19.574246 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:19.576820 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.576800 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.579254 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.579225 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bq5lz\"" Jul 20 04:59:19.579370 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.579278 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 20 04:59:19.579562 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.579530 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 20 04:59:19.588535 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.588514 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gzxvz" Jul 20 04:59:19.600716 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.600693 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gzxvz" Jul 20 04:59:19.652560 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.652508 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-33.ec2.internal" event={"ID":"a8c6bf12d5d282ff52e7d38919e9afae","Type":"ContainerStarted","Data":"73d9ebab32d3319233d9f49a4bb9f96b887789a58802e8e0078295665c115f89"} Jul 20 04:59:19.653703 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.653674 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal" event={"ID":"c80a066a6bda5fad4b7c26559d43b6cf","Type":"ContainerStarted","Data":"678f2688ab63e378e1ed3497ec240cc4a8cbb33b6f32fc9a5eb54a342616ba1c"} Jul 20 04:59:19.653923 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.653907 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 20 04:59:19.670474 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670448 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-kubernetes\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.670597 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670481 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xbpw\" (UniqueName: \"kubernetes.io/projected/1edae688-9ce1-49e7-9a37-5d782b643b36-kube-api-access-7xbpw\") pod \"node-resolver-js7gh\" (UID: \"1edae688-9ce1-49e7-9a37-5d782b643b36\") " pod="openshift-dns/node-resolver-js7gh" Jul 20 04:59:19.670597 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670506 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w5kv\" (UniqueName: \"kubernetes.io/projected/42007563-e764-4689-a28f-872b48cd1e8b-kube-api-access-7w5kv\") pod \"node-ca-p29tq\" (UID: \"42007563-e764-4689-a28f-872b48cd1e8b\") " pod="openshift-image-registry/node-ca-p29tq" Jul 20 04:59:19.670597 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670575 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-hostroot\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.670770 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670619 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-systemd-units\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.670770 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670654 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-run-ovn\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.670770 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670679 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-sysctl-d\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.670770 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670704 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-lib-modules\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.670770 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670758 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-run-k8s-cni-cncf-io\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.671023 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670781 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-etc-kubernetes\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.671023 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670802 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-etc-openvswitch\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.671023 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670862 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-node-log\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.671023 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670899 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-sysctl-conf\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.671023 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.670995 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rj7z\" (UniqueName: \"kubernetes.io/projected/06a59c1d-4c72-423f-9e8d-27db0a919dba-kube-api-access-9rj7z\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:19.671285 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671255 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-kubelet-config\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:19.671419 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671294 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-dbus\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:19.671419 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671312 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-var-lib-openvswitch\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.671419 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671331 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.671419 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671352 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx8ml\" (UniqueName: \"kubernetes.io/projected/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-kube-api-access-vx8ml\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.671419 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671382 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1edae688-9ce1-49e7-9a37-5d782b643b36-tmp-dir\") pod \"node-resolver-js7gh\" (UID: \"1edae688-9ce1-49e7-9a37-5d782b643b36\") " pod="openshift-dns/node-resolver-js7gh" Jul 20 04:59:19.671419 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671411 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-sys-fs\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.671734 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671435 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-run-netns\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.671734 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671455 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-run-openvswitch\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.671734 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671477 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-system-cni-dir\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.671734 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671503 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-multus-cni-dir\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.671734 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671528 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0811f4e-d3ae-47c3-a490-a6c9c2db3010-agent-certs\") pod \"konnectivity-agent-4ljhn\" (UID: \"f0811f4e-d3ae-47c3-a490-a6c9c2db3010\") " pod="kube-system/konnectivity-agent-4ljhn" Jul 20 04:59:19.671734 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671551 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-sys-fs\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.671734 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671569 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxzbx\" (UniqueName: \"kubernetes.io/projected/461b67c2-73dc-4b07-9b52-f102773c0aef-kube-api-access-kxzbx\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.671734 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671609 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-host\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.671734 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671636 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-system-cni-dir\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.671734 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671663 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-cnibin\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.671734 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671692 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.671734 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671702 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1edae688-9ce1-49e7-9a37-5d782b643b36-tmp-dir\") pod \"node-resolver-js7gh\" (UID: \"1edae688-9ce1-49e7-9a37-5d782b643b36\") " pod="openshift-dns/node-resolver-js7gh" Jul 20 04:59:19.671734 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671730 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-multus-daemon-config\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671808 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbjb6\" (UniqueName: \"kubernetes.io/projected/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-kube-api-access-zbjb6\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671835 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-slash\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671849 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671870 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-cni-netd\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671899 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/461b67c2-73dc-4b07-9b52-f102773c0aef-ovn-node-metrics-cert\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671920 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-registration-dir\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671939 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7x4f\" (UniqueName: \"kubernetes.io/projected/6a61c804-fec7-4b33-96d5-630b09a1eb30-kube-api-access-n7x4f\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671955 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-cni-binary-copy\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671988 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-multus-conf-dir\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.671997 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-registration-dir\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672028 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0811f4e-d3ae-47c3-a490-a6c9c2db3010-konnectivity-ca\") pod \"konnectivity-agent-4ljhn\" (UID: \"f0811f4e-d3ae-47c3-a490-a6c9c2db3010\") " pod="kube-system/konnectivity-agent-4ljhn" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672060 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-run-ovn-kubernetes\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672085 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/461b67c2-73dc-4b07-9b52-f102773c0aef-ovnkube-script-lib\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672110 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-os-release\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672133 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-socket-dir\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.672403 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672153 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-device-dir\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672171 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-multus-socket-dir-parent\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672194 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-run-multus-certs\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672228 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-device-dir\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672226 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-log-socket\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672270 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/461b67c2-73dc-4b07-9b52-f102773c0aef-env-overrides\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672297 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-systemd\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672321 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-tuned\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672350 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brzdv\" (UniqueName: \"kubernetes.io/projected/f7fe0af3-add4-47d8-81db-87b049070630-kube-api-access-brzdv\") pod \"iptables-alerter-2zdwn\" (UID: \"f7fe0af3-add4-47d8-81db-87b049070630\") " pod="openshift-network-operator/iptables-alerter-2zdwn" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672269 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-socket-dir\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672402 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/42007563-e764-4689-a28f-872b48cd1e8b-serviceca\") pod \"node-ca-p29tq\" (UID: \"42007563-e764-4689-a28f-872b48cd1e8b\") " pod="openshift-image-registry/node-ca-p29tq" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672428 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672446 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672517 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a61c804-fec7-4b33-96d5-630b09a1eb30-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672623 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-kubelet\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672651 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xncf\" (UniqueName: \"kubernetes.io/projected/1e0e744e-13f1-42d9-ad84-ff372341ff1e-kube-api-access-2xncf\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672670 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.673300 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672687 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-cnibin\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672708 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-run-netns\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:19.672715 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672732 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/461b67c2-73dc-4b07-9b52-f102773c0aef-ovnkube-config\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672762 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7fe0af3-add4-47d8-81db-87b049070630-iptables-alerter-script\") pod \"iptables-alerter-2zdwn\" (UID: \"f7fe0af3-add4-47d8-81db-87b049070630\") " pod="openshift-network-operator/iptables-alerter-2zdwn" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:19.672785 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs podName:06a59c1d-4c72-423f-9e8d-27db0a919dba nodeName:}" failed. No retries permitted until 2026-07-20 04:59:20.172762135 +0000 UTC m=+3.219289040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs") pod "network-metrics-daemon-gnmjm" (UID: "06a59c1d-4c72-423f-9e8d-27db0a919dba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672818 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-var-lib-kubelet\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672845 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-cni-bin\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672870 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-sysconfig\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672897 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672936 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-var-lib-cni-bin\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672962 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-modprobe-d\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.672996 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/42007563-e764-4689-a28f-872b48cd1e8b-serviceca\") pod \"node-ca-p29tq\" (UID: \"42007563-e764-4689-a28f-872b48cd1e8b\") " pod="openshift-image-registry/node-ca-p29tq" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673011 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-sys\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673038 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-cni-binary-copy\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673064 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-os-release\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.674179 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673097 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-var-lib-cni-multus\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673118 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl7dt\" (UniqueName: \"kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt\") pod \"network-check-target-nwhn4\" (UID: \"1d8591ce-a880-4fa7-9792-313af6b166e0\") " pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673144 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1edae688-9ce1-49e7-9a37-5d782b643b36-hosts-file\") pod \"node-resolver-js7gh\" (UID: \"1edae688-9ce1-49e7-9a37-5d782b643b36\") " pod="openshift-dns/node-resolver-js7gh" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673170 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42007563-e764-4689-a28f-872b48cd1e8b-host\") pod \"node-ca-p29tq\" (UID: \"42007563-e764-4689-a28f-872b48cd1e8b\") " pod="openshift-image-registry/node-ca-p29tq" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673190 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673209 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-var-lib-kubelet\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673221 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1edae688-9ce1-49e7-9a37-5d782b643b36-hosts-file\") pod \"node-resolver-js7gh\" (UID: \"1edae688-9ce1-49e7-9a37-5d782b643b36\") " pod="openshift-dns/node-resolver-js7gh" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673225 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e0e744e-13f1-42d9-ad84-ff372341ff1e-tmp\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673269 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42007563-e764-4689-a28f-872b48cd1e8b-host\") pod \"node-ca-p29tq\" (UID: \"42007563-e764-4689-a28f-872b48cd1e8b\") " pod="openshift-image-registry/node-ca-p29tq" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673287 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7fe0af3-add4-47d8-81db-87b049070630-host-slash\") pod \"iptables-alerter-2zdwn\" (UID: \"f7fe0af3-add4-47d8-81db-87b049070630\") " pod="openshift-network-operator/iptables-alerter-2zdwn" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673321 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673340 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7fe0af3-add4-47d8-81db-87b049070630-iptables-alerter-script\") pod \"iptables-alerter-2zdwn\" (UID: \"f7fe0af3-add4-47d8-81db-87b049070630\") " pod="openshift-network-operator/iptables-alerter-2zdwn" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673350 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-run\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673357 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7fe0af3-add4-47d8-81db-87b049070630-host-slash\") pod \"iptables-alerter-2zdwn\" (UID: \"f7fe0af3-add4-47d8-81db-87b049070630\") " pod="openshift-network-operator/iptables-alerter-2zdwn" Jul 20 04:59:19.675062 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.673376 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-run-systemd\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.683937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.683911 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 20 04:59:19.688154 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.688055 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7x4f\" (UniqueName: \"kubernetes.io/projected/6a61c804-fec7-4b33-96d5-630b09a1eb30-kube-api-access-n7x4f\") pod \"aws-ebs-csi-driver-node-p2lqj\" (UID: \"6a61c804-fec7-4b33-96d5-630b09a1eb30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.688154 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.688061 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xbpw\" (UniqueName: \"kubernetes.io/projected/1edae688-9ce1-49e7-9a37-5d782b643b36-kube-api-access-7xbpw\") pod \"node-resolver-js7gh\" (UID: \"1edae688-9ce1-49e7-9a37-5d782b643b36\") " pod="openshift-dns/node-resolver-js7gh" Jul 20 04:59:19.688154 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.688126 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rj7z\" (UniqueName: \"kubernetes.io/projected/06a59c1d-4c72-423f-9e8d-27db0a919dba-kube-api-access-9rj7z\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:19.688392 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.688175 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w5kv\" (UniqueName: \"kubernetes.io/projected/42007563-e764-4689-a28f-872b48cd1e8b-kube-api-access-7w5kv\") pod \"node-ca-p29tq\" (UID: \"42007563-e764-4689-a28f-872b48cd1e8b\") " pod="openshift-image-registry/node-ca-p29tq" Jul 20 04:59:19.688640 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.688612 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brzdv\" (UniqueName: \"kubernetes.io/projected/f7fe0af3-add4-47d8-81db-87b049070630-kube-api-access-brzdv\") pod \"iptables-alerter-2zdwn\" (UID: \"f7fe0af3-add4-47d8-81db-87b049070630\") " pod="openshift-network-operator/iptables-alerter-2zdwn" Jul 20 04:59:19.774676 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774636 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-dbus\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:19.774676 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774681 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-var-lib-openvswitch\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.774932 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774712 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.774932 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774739 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8ml\" (UniqueName: \"kubernetes.io/projected/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-kube-api-access-vx8ml\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.774932 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774769 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-run-netns\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.774932 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774772 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-var-lib-openvswitch\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.774932 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774786 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-run-openvswitch\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.774932 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774810 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-system-cni-dir\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.774932 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774834 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-run-netns\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.774932 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774835 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-multus-cni-dir\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.774932 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774888 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0811f4e-d3ae-47c3-a490-a6c9c2db3010-agent-certs\") pod \"konnectivity-agent-4ljhn\" (UID: \"f0811f4e-d3ae-47c3-a490-a6c9c2db3010\") " pod="kube-system/konnectivity-agent-4ljhn" Jul 20 04:59:19.774932 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774915 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxzbx\" (UniqueName: \"kubernetes.io/projected/461b67c2-73dc-4b07-9b52-f102773c0aef-kube-api-access-kxzbx\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774942 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-host\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774943 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-dbus\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.774961 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-multus-cni-dir\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775003 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-system-cni-dir\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775034 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-cnibin\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775065 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-multus-daemon-config\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775077 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-system-cni-dir\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775098 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbjb6\" (UniqueName: \"kubernetes.io/projected/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-kube-api-access-zbjb6\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775107 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-system-cni-dir\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775123 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-slash\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775139 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-host\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775149 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-cni-netd\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775196 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/461b67c2-73dc-4b07-9b52-f102773c0aef-ovn-node-metrics-cert\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775227 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-cni-binary-copy\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775251 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-multus-conf-dir\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775277 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0811f4e-d3ae-47c3-a490-a6c9c2db3010-konnectivity-ca\") pod \"konnectivity-agent-4ljhn\" (UID: \"f0811f4e-d3ae-47c3-a490-a6c9c2db3010\") " pod="kube-system/konnectivity-agent-4ljhn" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775304 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-run-ovn-kubernetes\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.775326 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775331 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/461b67c2-73dc-4b07-9b52-f102773c0aef-ovnkube-script-lib\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775361 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-os-release\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775372 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-cnibin\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775394 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-multus-socket-dir-parent\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775420 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-run-multus-certs\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775445 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-log-socket\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775469 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/461b67c2-73dc-4b07-9b52-f102773c0aef-env-overrides\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775495 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-systemd\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775518 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-tuned\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775561 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-kubelet\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775587 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xncf\" (UniqueName: \"kubernetes.io/projected/1e0e744e-13f1-42d9-ad84-ff372341ff1e-kube-api-access-2xncf\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775613 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775639 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-cnibin\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775668 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-run-netns\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775693 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/461b67c2-73dc-4b07-9b52-f102773c0aef-ovnkube-config\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775716 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-multus-daemon-config\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775779 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-var-lib-kubelet\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.776177 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775720 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-var-lib-kubelet\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775825 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-cni-bin\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775854 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-sysconfig\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775883 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775912 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-var-lib-cni-bin\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775938 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-modprobe-d\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775943 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-slash\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.775964 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-sys\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776007 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-sys\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776038 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-cni-binary-copy\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776067 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-os-release\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776077 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0811f4e-d3ae-47c3-a490-a6c9c2db3010-konnectivity-ca\") pod \"konnectivity-agent-4ljhn\" (UID: \"f0811f4e-d3ae-47c3-a490-a6c9c2db3010\") " pod="kube-system/konnectivity-agent-4ljhn" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776084 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776102 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-var-lib-cni-multus\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776115 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-sysconfig\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776136 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl7dt\" (UniqueName: \"kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt\") pod \"network-check-target-nwhn4\" (UID: \"1d8591ce-a880-4fa7-9792-313af6b166e0\") " pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776145 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-systemd\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.777014 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776159 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-log-socket\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776158 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-var-lib-cni-bin\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776169 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776193 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-multus-conf-dir\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776203 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-var-lib-kubelet\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776224 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-os-release\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776230 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e0e744e-13f1-42d9-ad84-ff372341ff1e-tmp\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776241 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-modprobe-d\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776271 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776300 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-run\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776323 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-run-multus-certs\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776326 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-run-systemd\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776348 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776361 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-kubernetes\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776388 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/461b67c2-73dc-4b07-9b52-f102773c0aef-ovnkube-script-lib\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776406 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-multus-socket-dir-parent\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776392 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-hostroot\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776423 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-hostroot\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.777937 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776473 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-os-release\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776478 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-cnibin\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776070 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-cni-bin\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776478 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/461b67c2-73dc-4b07-9b52-f102773c0aef-env-overrides\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:19.776486 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776520 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-systemd-units\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776547 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-run-ovn\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776578 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-run-ovn\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:19.776596 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret podName:a9dd925c-ebe0-4584-a59a-bcc23cb8a69b nodeName:}" failed. No retries permitted until 2026-07-20 04:59:20.276546856 +0000 UTC m=+3.323073758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret") pod "global-pull-secret-syncer-dh2cl" (UID: "a9dd925c-ebe0-4584-a59a-bcc23cb8a69b") : object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776627 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-run\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776638 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-var-lib-cni-multus\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776665 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-systemd-units\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776678 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-cni-netd\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776724 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-kubernetes\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776363 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-run-systemd\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776769 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-cni-binary-copy\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776924 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.778869 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776927 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-sysctl-d\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776953 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/461b67c2-73dc-4b07-9b52-f102773c0aef-ovnkube-config\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.776978 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-lib-modules\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777011 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-run-k8s-cni-cncf-io\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777025 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-sysctl-d\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777052 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-run-netns\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777060 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-etc-kubernetes\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777092 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-cni-binary-copy\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777106 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-host-run-k8s-cni-cncf-io\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777116 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777140 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-run-ovn-kubernetes\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777168 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-lib-modules\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777159 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-var-lib-kubelet\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777190 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-host-kubelet\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777213 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-etc-kubernetes\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777224 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-etc-openvswitch\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777252 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-node-log\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777276 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-sysctl-conf\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.780302 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777303 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-kubelet-config\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:19.781191 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777358 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-node-log\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.781191 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777358 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-etc-openvswitch\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.781191 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777392 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-kubelet-config\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:19.781191 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777454 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-sysctl-conf\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.781191 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.777484 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/461b67c2-73dc-4b07-9b52-f102773c0aef-run-openvswitch\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.781191 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.779094 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e0e744e-13f1-42d9-ad84-ff372341ff1e-etc-tuned\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.781191 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.779715 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/461b67c2-73dc-4b07-9b52-f102773c0aef-ovn-node-metrics-cert\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.782516 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.782485 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0811f4e-d3ae-47c3-a490-a6c9c2db3010-agent-certs\") pod \"konnectivity-agent-4ljhn\" (UID: \"f0811f4e-d3ae-47c3-a490-a6c9c2db3010\") " pod="kube-system/konnectivity-agent-4ljhn" Jul 20 04:59:19.783107 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.783084 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e0e744e-13f1-42d9-ad84-ff372341ff1e-tmp\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.783776 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:19.783405 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 04:59:19.783776 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:19.783457 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 04:59:19.783776 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:19.783473 2576 projected.go:196] Error preparing data for projected volume kube-api-access-rl7dt for pod openshift-network-diagnostics/network-check-target-nwhn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:19.783776 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:19.783551 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt podName:1d8591ce-a880-4fa7-9792-313af6b166e0 nodeName:}" failed. No retries permitted until 2026-07-20 04:59:20.283526017 +0000 UTC m=+3.330052990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rl7dt" (UniqueName: "kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt") pod "network-check-target-nwhn4" (UID: "1d8591ce-a880-4fa7-9792-313af6b166e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:19.784095 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.783827 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbjb6\" (UniqueName: \"kubernetes.io/projected/dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe-kube-api-access-zbjb6\") pod \"multus-lpkbr\" (UID: \"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe\") " pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.785090 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.785050 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx8ml\" (UniqueName: \"kubernetes.io/projected/f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1-kube-api-access-vx8ml\") pod \"multus-additional-cni-plugins-lhssz\" (UID: \"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1\") " pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:19.785200 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.785115 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxzbx\" (UniqueName: \"kubernetes.io/projected/461b67c2-73dc-4b07-9b52-f102773c0aef-kube-api-access-kxzbx\") pod \"ovnkube-node-85lxq\" (UID: \"461b67c2-73dc-4b07-9b52-f102773c0aef\") " pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.787251 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.787230 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xncf\" (UniqueName: \"kubernetes.io/projected/1e0e744e-13f1-42d9-ad84-ff372341ff1e-kube-api-access-2xncf\") pod \"tuned-lffh5\" (UID: \"1e0e744e-13f1-42d9-ad84-ff372341ff1e\") " pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.864030 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.863988 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2zdwn" Jul 20 04:59:19.871123 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.871100 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" Jul 20 04:59:19.881836 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.881816 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-js7gh" Jul 20 04:59:19.888783 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.888758 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p29tq" Jul 20 04:59:19.896824 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.896803 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lpkbr" Jul 20 04:59:19.903889 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.903868 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:19.912933 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.912912 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4ljhn" Jul 20 04:59:19.919112 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.919085 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lffh5" Jul 20 04:59:19.925082 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:19.925059 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lhssz" Jul 20 04:59:20.008435 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:20.008407 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 04:59:20.181213 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:20.181122 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:20.181379 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:20.181276 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:20.181379 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:20.181367 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs podName:06a59c1d-4c72-423f-9e8d-27db0a919dba nodeName:}" failed. No retries permitted until 2026-07-20 04:59:21.18134214 +0000 UTC m=+4.227869027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs") pod "network-metrics-daemon-gnmjm" (UID: "06a59c1d-4c72-423f-9e8d-27db0a919dba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:20.281676 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:20.281640 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:20.281829 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:20.281765 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:20.281829 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:20.281828 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret podName:a9dd925c-ebe0-4584-a59a-bcc23cb8a69b nodeName:}" failed. No retries permitted until 2026-07-20 04:59:21.28180992 +0000 UTC m=+4.328336817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret") pod "global-pull-secret-syncer-dh2cl" (UID: "a9dd925c-ebe0-4584-a59a-bcc23cb8a69b") : object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:20.383178 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:20.383132 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl7dt\" (UniqueName: \"kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt\") pod \"network-check-target-nwhn4\" (UID: \"1d8591ce-a880-4fa7-9792-313af6b166e0\") " pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:20.383383 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:20.383334 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 04:59:20.383383 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:20.383360 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 04:59:20.383383 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:20.383374 2576 projected.go:196] Error preparing data for projected volume kube-api-access-rl7dt for pod openshift-network-diagnostics/network-check-target-nwhn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:20.383560 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:20.383446 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt podName:1d8591ce-a880-4fa7-9792-313af6b166e0 nodeName:}" failed. No retries permitted until 2026-07-20 04:59:21.38342465 +0000 UTC m=+4.429951560 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rl7dt" (UniqueName: "kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt") pod "network-check-target-nwhn4" (UID: "1d8591ce-a880-4fa7-9792-313af6b166e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:20.604973 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:20.604835 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 04:54:19 +0000 UTC" deadline="2028-03-22 22:16:25.40379902 +0000 UTC" Jul 20 04:59:20.604973 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:20.604903 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14681h17m4.798900597s" Jul 20 04:59:20.657395 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:20.657083 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42007563_e764_4689_a28f_872b48cd1e8b.slice/crio-e472d4205bb35d9183222939245b8fa4a969c066443de39b2e57cf3e48aaff49 WatchSource:0}: Error finding container e472d4205bb35d9183222939245b8fa4a969c066443de39b2e57cf3e48aaff49: Status 404 returned error can't find the container with id e472d4205bb35d9183222939245b8fa4a969c066443de39b2e57cf3e48aaff49 Jul 20 04:59:20.658840 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:20.658812 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0e744e_13f1_42d9_ad84_ff372341ff1e.slice/crio-23a9a0ed81a57b4920bc0e0a8d24001154abc78e721d56c60c9fb6d5ce2e4e64 WatchSource:0}: Error finding container 23a9a0ed81a57b4920bc0e0a8d24001154abc78e721d56c60c9fb6d5ce2e4e64: Status 404 returned error can't find the container with id 23a9a0ed81a57b4920bc0e0a8d24001154abc78e721d56c60c9fb6d5ce2e4e64 Jul 20 04:59:20.660092 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:20.660062 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcad9ac3_69d3_4c84_9f0f_dc1f45e86bfe.slice/crio-0316b2f3af750f371732fb89786e7a5ed6fa7d0b5b1d084cb2df27b2ae284257 WatchSource:0}: Error finding container 0316b2f3af750f371732fb89786e7a5ed6fa7d0b5b1d084cb2df27b2ae284257: Status 404 returned error can't find the container with id 0316b2f3af750f371732fb89786e7a5ed6fa7d0b5b1d084cb2df27b2ae284257 Jul 20 04:59:20.661131 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:20.661076 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a61c804_fec7_4b33_96d5_630b09a1eb30.slice/crio-f0ebc3da049d35b5fe0f04251cbb87c8e774367c0077350cd1a148e1b3c98934 WatchSource:0}: Error finding container f0ebc3da049d35b5fe0f04251cbb87c8e774367c0077350cd1a148e1b3c98934: Status 404 returned error can't find the container with id f0ebc3da049d35b5fe0f04251cbb87c8e774367c0077350cd1a148e1b3c98934 Jul 20 04:59:20.664231 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:20.663942 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7fe0af3_add4_47d8_81db_87b049070630.slice/crio-6e7d4e16fd09482c23106ddf446ee6b52911b0aebc8c83c90dd64ec560305331 WatchSource:0}: Error finding container 6e7d4e16fd09482c23106ddf446ee6b52911b0aebc8c83c90dd64ec560305331: Status 404 returned error can't find the container with id 6e7d4e16fd09482c23106ddf446ee6b52911b0aebc8c83c90dd64ec560305331 Jul 20 04:59:20.665169 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:20.665144 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1edae688_9ce1_49e7_9a37_5d782b643b36.slice/crio-6f7cc0b4b5c0898c644cc02293c0d929afedce1429411dfc3a201615057aa868 WatchSource:0}: Error finding container 6f7cc0b4b5c0898c644cc02293c0d929afedce1429411dfc3a201615057aa868: Status 404 returned error can't find the container with id 6f7cc0b4b5c0898c644cc02293c0d929afedce1429411dfc3a201615057aa868 Jul 20 04:59:20.687707 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:20.687674 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf954ae54_6ba3_4d8c_a6e1_180fa75fc3a1.slice/crio-cf25ea76a5de4f8c060cc4a8a5b392c14d5d567080569ac5ff114366fd69f8ec WatchSource:0}: Error finding container cf25ea76a5de4f8c060cc4a8a5b392c14d5d567080569ac5ff114366fd69f8ec: Status 404 returned error can't find the container with id cf25ea76a5de4f8c060cc4a8a5b392c14d5d567080569ac5ff114366fd69f8ec Jul 20 04:59:20.688694 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:20.688672 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0811f4e_d3ae_47c3_a490_a6c9c2db3010.slice/crio-a369e1630df48a6648a5dc1b9b9ebabc108d357614ef32ea76804113726a3424 WatchSource:0}: Error finding container a369e1630df48a6648a5dc1b9b9ebabc108d357614ef32ea76804113726a3424: Status 404 returned error can't find the container with id a369e1630df48a6648a5dc1b9b9ebabc108d357614ef32ea76804113726a3424 Jul 20 04:59:20.689757 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:20.689739 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod461b67c2_73dc_4b07_9b52_f102773c0aef.slice/crio-50359986ccfc6d3d219d3734f60214391b70bca6ccad7ed151c235170a538a74 WatchSource:0}: Error finding container 50359986ccfc6d3d219d3734f60214391b70bca6ccad7ed151c235170a538a74: Status 404 returned error can't find the container with id 50359986ccfc6d3d219d3734f60214391b70bca6ccad7ed151c235170a538a74 Jul 20 04:59:21.189832 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.189425 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:21.190070 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:21.190049 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:21.190139 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:21.190131 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs podName:06a59c1d-4c72-423f-9e8d-27db0a919dba nodeName:}" failed. No retries permitted until 2026-07-20 04:59:23.190108145 +0000 UTC m=+6.236635036 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs") pod "network-metrics-daemon-gnmjm" (UID: "06a59c1d-4c72-423f-9e8d-27db0a919dba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:21.291117 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.291077 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:21.291296 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:21.291252 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:21.291371 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:21.291317 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret podName:a9dd925c-ebe0-4584-a59a-bcc23cb8a69b nodeName:}" failed. No retries permitted until 2026-07-20 04:59:23.291297507 +0000 UTC m=+6.337824395 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret") pod "global-pull-secret-syncer-dh2cl" (UID: "a9dd925c-ebe0-4584-a59a-bcc23cb8a69b") : object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:21.393110 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.392461 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl7dt\" (UniqueName: \"kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt\") pod \"network-check-target-nwhn4\" (UID: \"1d8591ce-a880-4fa7-9792-313af6b166e0\") " pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:21.393110 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:21.392631 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 04:59:21.393110 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:21.392653 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 04:59:21.393110 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:21.392666 2576 projected.go:196] Error preparing data for projected volume kube-api-access-rl7dt for pod openshift-network-diagnostics/network-check-target-nwhn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:21.393110 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:21.392728 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt podName:1d8591ce-a880-4fa7-9792-313af6b166e0 nodeName:}" failed. No retries permitted until 2026-07-20 04:59:23.392708769 +0000 UTC m=+6.439235675 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rl7dt" (UniqueName: "kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt") pod "network-check-target-nwhn4" (UID: "1d8591ce-a880-4fa7-9792-313af6b166e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:21.605953 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.605854 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 04:54:19 +0000 UTC" deadline="2028-01-17 20:11:06.412396391 +0000 UTC" Jul 20 04:59:21.605953 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.605898 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13119h11m44.806502858s" Jul 20 04:59:21.652446 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.651555 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:21.652446 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:21.651693 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:21.652446 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.652139 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:21.652446 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:21.652247 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:21.652446 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.652310 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:21.652446 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:21.652390 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:21.667898 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.667812 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-js7gh" event={"ID":"1edae688-9ce1-49e7-9a37-5d782b643b36","Type":"ContainerStarted","Data":"6f7cc0b4b5c0898c644cc02293c0d929afedce1429411dfc3a201615057aa868"} Jul 20 04:59:21.669815 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.669722 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" event={"ID":"461b67c2-73dc-4b07-9b52-f102773c0aef","Type":"ContainerStarted","Data":"50359986ccfc6d3d219d3734f60214391b70bca6ccad7ed151c235170a538a74"} Jul 20 04:59:21.677201 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.677173 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2zdwn" event={"ID":"f7fe0af3-add4-47d8-81db-87b049070630","Type":"ContainerStarted","Data":"6e7d4e16fd09482c23106ddf446ee6b52911b0aebc8c83c90dd64ec560305331"} Jul 20 04:59:21.689609 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.689577 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" event={"ID":"6a61c804-fec7-4b33-96d5-630b09a1eb30","Type":"ContainerStarted","Data":"f0ebc3da049d35b5fe0f04251cbb87c8e774367c0077350cd1a148e1b3c98934"} Jul 20 04:59:21.700907 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.700861 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lffh5" event={"ID":"1e0e744e-13f1-42d9-ad84-ff372341ff1e","Type":"ContainerStarted","Data":"23a9a0ed81a57b4920bc0e0a8d24001154abc78e721d56c60c9fb6d5ce2e4e64"} Jul 20 04:59:21.720415 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.720098 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-33.ec2.internal" event={"ID":"a8c6bf12d5d282ff52e7d38919e9afae","Type":"ContainerStarted","Data":"34b17e964c8375c42dfe78cef99da30e5c0b54d0febc5fad86e689380a2cce32"} Jul 20 04:59:21.739209 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.738500 2576 generic.go:356] "Generic (PLEG): container finished" podID="c80a066a6bda5fad4b7c26559d43b6cf" containerID="7846226d3deb424f8c49f5ba5d4c34aacbd48d82ed04adc8ba54380a334cc990" exitCode=0 Jul 20 04:59:21.739209 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.739094 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal" event={"ID":"c80a066a6bda5fad4b7c26559d43b6cf","Type":"ContainerDied","Data":"7846226d3deb424f8c49f5ba5d4c34aacbd48d82ed04adc8ba54380a334cc990"} Jul 20 04:59:21.747483 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.747451 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4ljhn" event={"ID":"f0811f4e-d3ae-47c3-a490-a6c9c2db3010","Type":"ContainerStarted","Data":"a369e1630df48a6648a5dc1b9b9ebabc108d357614ef32ea76804113726a3424"} Jul 20 04:59:21.758811 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.755930 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-33.ec2.internal" podStartSLOduration=2.755910296 podStartE2EDuration="2.755910296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 04:59:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 04:59:21.736551465 +0000 UTC m=+4.783078369" watchObservedRunningTime="2026-07-20 04:59:21.755910296 +0000 UTC m=+4.802437211" Jul 20 04:59:21.771617 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.771235 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhssz" event={"ID":"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1","Type":"ContainerStarted","Data":"cf25ea76a5de4f8c060cc4a8a5b392c14d5d567080569ac5ff114366fd69f8ec"} Jul 20 04:59:21.784998 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.784961 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lpkbr" event={"ID":"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe","Type":"ContainerStarted","Data":"0316b2f3af750f371732fb89786e7a5ed6fa7d0b5b1d084cb2df27b2ae284257"} Jul 20 04:59:21.789010 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:21.788977 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p29tq" event={"ID":"42007563-e764-4689-a28f-872b48cd1e8b","Type":"ContainerStarted","Data":"e472d4205bb35d9183222939245b8fa4a969c066443de39b2e57cf3e48aaff49"} Jul 20 04:59:22.813950 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:22.813910 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal" event={"ID":"c80a066a6bda5fad4b7c26559d43b6cf","Type":"ContainerStarted","Data":"809c04db923bddc27b7e145031de3d32d27e74e04034cc0f49aea567290a5151"} Jul 20 04:59:22.829507 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:22.829389 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-33.ec2.internal" podStartSLOduration=3.829368805 podStartE2EDuration="3.829368805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 04:59:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 04:59:22.828705172 +0000 UTC m=+5.875232112" watchObservedRunningTime="2026-07-20 04:59:22.829368805 +0000 UTC m=+5.875895719" Jul 20 04:59:23.206587 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:23.206473 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:23.206768 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:23.206675 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:23.206768 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:23.206748 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs podName:06a59c1d-4c72-423f-9e8d-27db0a919dba nodeName:}" failed. No retries permitted until 2026-07-20 04:59:27.206728463 +0000 UTC m=+10.253255365 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs") pod "network-metrics-daemon-gnmjm" (UID: "06a59c1d-4c72-423f-9e8d-27db0a919dba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:23.307766 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:23.307570 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:23.307766 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:23.307757 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:23.308153 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:23.307846 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret podName:a9dd925c-ebe0-4584-a59a-bcc23cb8a69b nodeName:}" failed. No retries permitted until 2026-07-20 04:59:27.307816914 +0000 UTC m=+10.354343806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret") pod "global-pull-secret-syncer-dh2cl" (UID: "a9dd925c-ebe0-4584-a59a-bcc23cb8a69b") : object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:23.408926 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:23.408884 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl7dt\" (UniqueName: \"kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt\") pod \"network-check-target-nwhn4\" (UID: \"1d8591ce-a880-4fa7-9792-313af6b166e0\") " pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:23.409315 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:23.409073 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 04:59:23.409315 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:23.409100 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 04:59:23.409315 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:23.409113 2576 projected.go:196] Error preparing data for projected volume kube-api-access-rl7dt for pod openshift-network-diagnostics/network-check-target-nwhn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:23.409315 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:23.409186 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt podName:1d8591ce-a880-4fa7-9792-313af6b166e0 nodeName:}" failed. No retries permitted until 2026-07-20 04:59:27.409158584 +0000 UTC m=+10.455685470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rl7dt" (UniqueName: "kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt") pod "network-check-target-nwhn4" (UID: "1d8591ce-a880-4fa7-9792-313af6b166e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:23.650260 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:23.650124 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:23.650426 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:23.650267 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:23.650695 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:23.650676 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:23.650816 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:23.650794 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:23.650877 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:23.650864 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:23.650970 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:23.650936 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:25.647275 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:25.647245 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:25.647711 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:25.647397 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:25.647914 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:25.647776 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:25.647914 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:25.647880 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:25.648128 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:25.648042 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:25.648189 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:25.648146 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:27.238558 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:27.238517 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:27.239037 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:27.238695 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:27.239037 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:27.238756 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs podName:06a59c1d-4c72-423f-9e8d-27db0a919dba nodeName:}" failed. No retries permitted until 2026-07-20 04:59:35.238736885 +0000 UTC m=+18.285263786 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs") pod "network-metrics-daemon-gnmjm" (UID: "06a59c1d-4c72-423f-9e8d-27db0a919dba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:27.339999 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:27.339886 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:27.340192 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:27.340044 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:27.340192 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:27.340108 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret podName:a9dd925c-ebe0-4584-a59a-bcc23cb8a69b nodeName:}" failed. No retries permitted until 2026-07-20 04:59:35.340090296 +0000 UTC m=+18.386617185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret") pod "global-pull-secret-syncer-dh2cl" (UID: "a9dd925c-ebe0-4584-a59a-bcc23cb8a69b") : object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:27.441345 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:27.441279 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl7dt\" (UniqueName: \"kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt\") pod \"network-check-target-nwhn4\" (UID: \"1d8591ce-a880-4fa7-9792-313af6b166e0\") " pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:27.441523 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:27.441431 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 04:59:27.441523 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:27.441451 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 04:59:27.441523 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:27.441463 2576 projected.go:196] Error preparing data for projected volume kube-api-access-rl7dt for pod openshift-network-diagnostics/network-check-target-nwhn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:27.441523 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:27.441521 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt podName:1d8591ce-a880-4fa7-9792-313af6b166e0 nodeName:}" failed. No retries permitted until 2026-07-20 04:59:35.44150251 +0000 UTC m=+18.488029397 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rl7dt" (UniqueName: "kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt") pod "network-check-target-nwhn4" (UID: "1d8591ce-a880-4fa7-9792-313af6b166e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:27.648630 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:27.648559 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:27.648781 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:27.648672 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:27.648894 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:27.648871 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:27.649068 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:27.649011 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:27.649068 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:27.649040 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:27.649197 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:27.649120 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:29.647771 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:29.647660 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:29.647771 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:29.647668 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:29.648330 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:29.647798 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:29.648330 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:29.647891 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:29.648330 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:29.647907 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:29.648330 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:29.648017 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:31.647621 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:31.647583 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:31.648097 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:31.647592 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:31.648097 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:31.647710 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:31.648097 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:31.647593 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:31.648097 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:31.647780 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:31.648097 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:31.647837 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:33.647206 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:33.647173 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:33.647668 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:33.647178 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:33.647668 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:33.647291 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:33.647668 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:33.647394 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:33.647668 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:33.647173 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:33.647668 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:33.647513 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:35.301333 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:35.301292 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:35.301797 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:35.301464 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:35.301797 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:35.301541 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs podName:06a59c1d-4c72-423f-9e8d-27db0a919dba nodeName:}" failed. No retries permitted until 2026-07-20 04:59:51.301521224 +0000 UTC m=+34.348048121 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs") pod "network-metrics-daemon-gnmjm" (UID: "06a59c1d-4c72-423f-9e8d-27db0a919dba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:35.402094 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:35.402059 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:35.402249 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:35.402168 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:35.402249 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:35.402218 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret podName:a9dd925c-ebe0-4584-a59a-bcc23cb8a69b nodeName:}" failed. No retries permitted until 2026-07-20 04:59:51.402205339 +0000 UTC m=+34.448732223 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret") pod "global-pull-secret-syncer-dh2cl" (UID: "a9dd925c-ebe0-4584-a59a-bcc23cb8a69b") : object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:35.502926 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:35.502891 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl7dt\" (UniqueName: \"kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt\") pod \"network-check-target-nwhn4\" (UID: \"1d8591ce-a880-4fa7-9792-313af6b166e0\") " pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:35.503129 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:35.503091 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 04:59:35.503129 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:35.503117 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 04:59:35.503129 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:35.503129 2576 projected.go:196] Error preparing data for projected volume kube-api-access-rl7dt for pod openshift-network-diagnostics/network-check-target-nwhn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:35.503263 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:35.503199 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt podName:1d8591ce-a880-4fa7-9792-313af6b166e0 nodeName:}" failed. No retries permitted until 2026-07-20 04:59:51.503177269 +0000 UTC m=+34.549704169 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rl7dt" (UniqueName: "kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt") pod "network-check-target-nwhn4" (UID: "1d8591ce-a880-4fa7-9792-313af6b166e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:35.647758 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:35.647673 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:35.647910 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:35.647673 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:35.647910 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:35.647829 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:35.647910 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:35.647673 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:35.647910 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:35.647896 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:35.648136 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:35.647999 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:37.647503 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:37.647472 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:37.648066 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:37.647568 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:37.648066 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:37.647582 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:37.648066 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:37.647642 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:37.648066 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:37.647681 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:37.648066 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:37.647760 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:39.647211 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:39.647174 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:39.647625 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:39.647174 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:39.647625 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:39.647283 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:39.647625 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:39.647297 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:39.647625 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:39.647419 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:39.647625 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:39.647505 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:40.851581 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.851171 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-js7gh" event={"ID":"1edae688-9ce1-49e7-9a37-5d782b643b36","Type":"ContainerStarted","Data":"313ed2cab48c88019c794e6a8d817a926cedaf9bcbcb4c4f0e4f780b0b130e49"} Jul 20 04:59:40.854225 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.854205 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 04:59:40.854617 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.854589 2576 generic.go:356] "Generic (PLEG): container finished" podID="461b67c2-73dc-4b07-9b52-f102773c0aef" containerID="1fa41295c42045e054b14128032def5ab63c30d9c649816ffa1a3ed9b09d94ac" exitCode=1 Jul 20 04:59:40.854722 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.854622 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" event={"ID":"461b67c2-73dc-4b07-9b52-f102773c0aef","Type":"ContainerStarted","Data":"113d9827fb0be4dc2a70a2869122c1f1d3e01cdc36a941c0f83a9ebc09887b71"} Jul 20 04:59:40.854722 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.854701 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" event={"ID":"461b67c2-73dc-4b07-9b52-f102773c0aef","Type":"ContainerStarted","Data":"31bacb2478a5b77980a7530b3472bfaaa672af96479c67e5da431eee62f4b30e"} Jul 20 04:59:40.854827 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.854750 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" event={"ID":"461b67c2-73dc-4b07-9b52-f102773c0aef","Type":"ContainerStarted","Data":"5b3e6a566e81458964326b66cc5810042aecceb806b2a6e0c9a34b2a9f136f34"} Jul 20 04:59:40.854827 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.854800 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" event={"ID":"461b67c2-73dc-4b07-9b52-f102773c0aef","Type":"ContainerStarted","Data":"9878e1dda4f9f9818174e152e9894adcfe31330482d38ada4c4ca1f0757de44b"} Jul 20 04:59:40.854928 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.854837 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" event={"ID":"461b67c2-73dc-4b07-9b52-f102773c0aef","Type":"ContainerDied","Data":"1fa41295c42045e054b14128032def5ab63c30d9c649816ffa1a3ed9b09d94ac"} Jul 20 04:59:40.854928 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.854883 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" event={"ID":"461b67c2-73dc-4b07-9b52-f102773c0aef","Type":"ContainerStarted","Data":"f397bdc7ef22397966c3ac0f8a9c980e398f435a64444285f7d671e2110efee8"} Jul 20 04:59:40.856393 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.856363 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" event={"ID":"6a61c804-fec7-4b33-96d5-630b09a1eb30","Type":"ContainerStarted","Data":"3d08748e0f83bcdf09b23d75b5f49a1d27f0f897651d12424787d40f33ed0cda"} Jul 20 04:59:40.857844 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.857820 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lffh5" event={"ID":"1e0e744e-13f1-42d9-ad84-ff372341ff1e","Type":"ContainerStarted","Data":"102cf51e5bf443d1c883e18903a55a284ddd6c3d0261bae263f0b40a2f312c30"} Jul 20 04:59:40.859389 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.859366 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4ljhn" event={"ID":"f0811f4e-d3ae-47c3-a490-a6c9c2db3010","Type":"ContainerStarted","Data":"d321a535af9973472ce9cc5dfdbf3651cfbe584bdbebd76ae29c4b980b470aa2"} Jul 20 04:59:40.860881 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.860862 2576 generic.go:356] "Generic (PLEG): container finished" podID="f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1" containerID="f59614878d137bb435cdf96d7e3cac571dfa6961d114f9eb191e78fb82249195" exitCode=0 Jul 20 04:59:40.861001 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.860934 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhssz" event={"ID":"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1","Type":"ContainerDied","Data":"f59614878d137bb435cdf96d7e3cac571dfa6961d114f9eb191e78fb82249195"} Jul 20 04:59:40.862544 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.862525 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lpkbr" event={"ID":"dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe","Type":"ContainerStarted","Data":"92b96db868258c1a09f3e748fefc73f37beb8da0712f38f0d4c4a988d2a0ad8b"} Jul 20 04:59:40.864066 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.864043 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p29tq" event={"ID":"42007563-e764-4689-a28f-872b48cd1e8b","Type":"ContainerStarted","Data":"10b7407dea4573792857185ec7632b78e8486a7761e7390cfde04866517628d1"} Jul 20 04:59:40.865711 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.865674 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-js7gh" podStartSLOduration=3.658048233 podStartE2EDuration="22.865662256s" totalImagesPullingTime="19.207614023s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 04:59:40.865272988 +0000 UTC m=+23.911799895" watchObservedRunningTime="2026-07-20 04:59:40.865662256 +0000 UTC m=+23.912189161" Jul 20 04:59:40.884685 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.884597 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lffh5" podStartSLOduration=3.6515355400000002 podStartE2EDuration="22.884581976s" totalImagesPullingTime="19.233046436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 04:59:40.884348094 +0000 UTC m=+23.930875002" watchObservedRunningTime="2026-07-20 04:59:40.884581976 +0000 UTC m=+23.931108879" Jul 20 04:59:40.920294 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.920241 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-p29tq" podStartSLOduration=3.690546831 podStartE2EDuration="22.920224149s" totalImagesPullingTime="19.229677318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 04:59:40.920028315 +0000 UTC m=+23.966555220" watchObservedRunningTime="2026-07-20 04:59:40.920224149 +0000 UTC m=+23.966751107" Jul 20 04:59:40.953223 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.953172 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4ljhn" podStartSLOduration=4.043306219 podStartE2EDuration="22.953154367s" totalImagesPullingTime="18.909848148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 04:59:40.932821822 +0000 UTC m=+23.979348749" watchObservedRunningTime="2026-07-20 04:59:40.953154367 +0000 UTC m=+23.999681277" Jul 20 04:59:40.953404 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.953368 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lpkbr" podStartSLOduration=3.6686479199999997 podStartE2EDuration="22.953362128s" totalImagesPullingTime="19.284714208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 04:59:40.953341109 +0000 UTC m=+23.999868015" watchObservedRunningTime="2026-07-20 04:59:40.953362128 +0000 UTC m=+23.999889033" Jul 20 04:59:40.990142 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:40.990107 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:41.070891 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:41.070832 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:41.121970 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:41.121920 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:41.440608 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:41.440553 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 20 04:59:41.635791 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:41.635650 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T04:59:41.440570891Z","UUID":"6f91c098-7375-4aa6-8447-3d6343d0f8cb","Handler":null,"Name":"","Endpoint":""} Jul 20 04:59:41.637365 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:41.637342 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 20 04:59:41.637495 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:41.637379 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 20 04:59:41.647016 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:41.646984 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:41.647141 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:41.646990 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:41.647141 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:41.647109 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:41.647141 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:41.647105 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:41.647349 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:41.647184 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:41.647349 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:41.647268 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:41.867917 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:41.867829 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2zdwn" event={"ID":"f7fe0af3-add4-47d8-81db-87b049070630","Type":"ContainerStarted","Data":"99fb3d8c6e62609faf583899105a7e0790dd4150913b1b90a792fd7f81de37c1"} Jul 20 04:59:41.869925 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:41.869887 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" event={"ID":"6a61c804-fec7-4b33-96d5-630b09a1eb30","Type":"ContainerStarted","Data":"793535116ea0bab235a0b154dd683e7e32850806d31e7549db37ab21319d7851"} Jul 20 04:59:41.881642 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:41.881566 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2zdwn" podStartSLOduration=4.673486787 podStartE2EDuration="23.88154404s" totalImagesPullingTime="19.208057253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 04:59:41.880806766 +0000 UTC m=+24.927333673" watchObservedRunningTime="2026-07-20 04:59:41.88154404 +0000 UTC m=+24.928070947" Jul 20 04:59:41.999163 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:41.999122 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:42.099154 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:42.099067 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:42.876497 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:42.876416 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 04:59:42.876925 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:42.876903 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" event={"ID":"461b67c2-73dc-4b07-9b52-f102773c0aef","Type":"ContainerStarted","Data":"5a22a77a3e395538e716dcb2ac065379e2820340d494e98efaac0e105d882200"} Jul 20 04:59:42.879766 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:42.879737 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" event={"ID":"6a61c804-fec7-4b33-96d5-630b09a1eb30","Type":"ContainerStarted","Data":"f692686ceab4165e2eab5842b58e0724dc997f08c2dbf2a0fc818c86d86a1856"} Jul 20 04:59:42.896613 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:42.896556 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2lqj" podStartSLOduration=3.32149212 podStartE2EDuration="24.896533958s" totalImagesPullingTime="21.575041838s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 04:59:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 04:59:42.895050874 +0000 UTC m=+25.941577780" watchObservedRunningTime="2026-07-20 04:59:42.896533958 +0000 UTC m=+25.943060867" Jul 20 04:59:43.647035 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:43.646998 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:43.647035 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:43.647034 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:43.647257 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:43.647035 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:43.647257 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:43.647114 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:43.647357 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:43.647249 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:43.647357 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:43.647346 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:44.191570 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:44.191386 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4ljhn" Jul 20 04:59:44.192204 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:44.192186 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4ljhn" Jul 20 04:59:44.884822 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:44.884786 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4ljhn" Jul 20 04:59:44.885295 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:44.885278 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4ljhn" Jul 20 04:59:44.937208 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:44.937177 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:45.647582 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:45.647340 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:45.648299 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:45.647340 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:45.648299 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:45.647622 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:45.648299 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:45.647400 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:45.648299 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:45.647686 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:45.648299 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:45.647795 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:45.668525 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:45.668496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:45.715224 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:45.715186 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:45.888050 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:45.888014 2576 generic.go:356] "Generic (PLEG): container finished" podID="f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1" containerID="35a36e95e70d7c229c5c004e1f37b8f7d70899ac6d3ff3578535d5ecade56095" exitCode=0 Jul 20 04:59:45.888234 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:45.888111 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhssz" event={"ID":"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1","Type":"ContainerDied","Data":"35a36e95e70d7c229c5c004e1f37b8f7d70899ac6d3ff3578535d5ecade56095"} Jul 20 04:59:45.891738 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:45.891720 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 04:59:45.892147 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:45.892128 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" event={"ID":"461b67c2-73dc-4b07-9b52-f102773c0aef","Type":"ContainerStarted","Data":"237e9a1e37b661740ce7301b15b7f8333d22390eba654de0c6162f565a24a294"} Jul 20 04:59:45.892439 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:45.892426 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:45.892514 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:45.892482 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:45.892616 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:45.892600 2576 scope.go:117] "RemoveContainer" containerID="1fa41295c42045e054b14128032def5ab63c30d9c649816ffa1a3ed9b09d94ac" Jul 20 04:59:45.908660 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:45.908639 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:46.901633 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:46.901603 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 04:59:46.902132 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:46.902102 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" event={"ID":"461b67c2-73dc-4b07-9b52-f102773c0aef","Type":"ContainerStarted","Data":"23461cb2a5d32fc0ccec875d3a90dc1bf99734d81762e920d960ccc75c5a30ca"} Jul 20 04:59:46.902895 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:46.902626 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:46.922144 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:46.922121 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 04:59:46.930406 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:46.930362 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" podStartSLOduration=9.647178156 podStartE2EDuration="28.930348768s" totalImagesPullingTime="19.283170612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 04:59:46.928924181 +0000 UTC m=+29.975451106" watchObservedRunningTime="2026-07-20 04:59:46.930348768 +0000 UTC m=+29.976875675" Jul 20 04:59:47.019356 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:47.019316 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:47.028411 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:47.028385 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dh2cl"] Jul 20 04:59:47.028564 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:47.028497 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:47.028610 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:47.028585 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:47.032009 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:47.031975 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nwhn4"] Jul 20 04:59:47.032171 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:47.032099 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:47.032224 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:47.032202 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:47.032603 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:47.032581 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gnmjm"] Jul 20 04:59:47.032712 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:47.032664 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:47.032785 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:47.032746 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:47.119337 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:47.119258 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:47.906896 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:47.906865 2576 generic.go:356] "Generic (PLEG): container finished" podID="f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1" containerID="d6302433694819c5eb19b5deeb69a0167115773968082ca88b05588cbc3f7ee0" exitCode=0 Jul 20 04:59:47.907384 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:47.906944 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhssz" event={"ID":"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1","Type":"ContainerDied","Data":"d6302433694819c5eb19b5deeb69a0167115773968082ca88b05588cbc3f7ee0"} Jul 20 04:59:48.647557 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:48.647517 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:48.647749 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:48.647519 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:48.647749 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:48.647651 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:48.647749 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:48.647520 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:48.647749 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:48.647713 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:48.647977 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:48.647766 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:48.912059 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:48.911969 2576 generic.go:356] "Generic (PLEG): container finished" podID="f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1" containerID="f90a77641be9306d2514afbaf0af85ea6e0a183620c495f36f23626079a56b66" exitCode=0 Jul 20 04:59:48.912469 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:48.912063 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhssz" event={"ID":"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1","Type":"ContainerDied","Data":"f90a77641be9306d2514afbaf0af85ea6e0a183620c495f36f23626079a56b66"} Jul 20 04:59:50.647322 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:50.647283 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:50.647322 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:50.647305 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:50.648097 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:50.647310 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:50.648097 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:50.647396 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:50.648097 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:50.647501 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:50.648097 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:50.647561 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:51.324840 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:51.324810 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:51.325039 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:51.324991 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:51.325112 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:51.325066 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs podName:06a59c1d-4c72-423f-9e8d-27db0a919dba nodeName:}" failed. No retries permitted until 2026-07-20 05:00:23.325045755 +0000 UTC m=+66.371572651 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs") pod "network-metrics-daemon-gnmjm" (UID: "06a59c1d-4c72-423f-9e8d-27db0a919dba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 04:59:51.393557 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:51.393525 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:51.425641 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:51.425612 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:51.428384 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:51.426049 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:51.428384 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:51.426144 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret podName:a9dd925c-ebe0-4584-a59a-bcc23cb8a69b nodeName:}" failed. No retries permitted until 2026-07-20 05:00:23.426123374 +0000 UTC m=+66.472650274 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret") pod "global-pull-secret-syncer-dh2cl" (UID: "a9dd925c-ebe0-4584-a59a-bcc23cb8a69b") : object "kube-system"/"original-pull-secret" not registered Jul 20 04:59:51.494470 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:51.494380 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:51.526803 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:51.526765 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl7dt\" (UniqueName: \"kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt\") pod \"network-check-target-nwhn4\" (UID: \"1d8591ce-a880-4fa7-9792-313af6b166e0\") " pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:51.527038 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:51.526942 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 04:59:51.527038 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:51.526984 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 04:59:51.527038 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:51.526998 2576 projected.go:196] Error preparing data for projected volume kube-api-access-rl7dt for pod openshift-network-diagnostics/network-check-target-nwhn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:51.527198 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:51.527059 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt podName:1d8591ce-a880-4fa7-9792-313af6b166e0 nodeName:}" failed. No retries permitted until 2026-07-20 05:00:23.527041923 +0000 UTC m=+66.573568809 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rl7dt" (UniqueName: "kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt") pod "network-check-target-nwhn4" (UID: "1d8591ce-a880-4fa7-9792-313af6b166e0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 04:59:52.647569 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.647487 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:52.648291 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.647622 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:52.648291 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.647640 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:52.648291 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:52.647732 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 04:59:52.648291 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:52.647618 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dh2cl" podUID="a9dd925c-ebe0-4584-a59a-bcc23cb8a69b" Jul 20 04:59:52.648291 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:52.647826 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwhn4" podUID="1d8591ce-a880-4fa7-9792-313af6b166e0" Jul 20 04:59:52.835098 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.835065 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-33.ec2.internal" event="NodeReady" Jul 20 04:59:52.835283 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.835236 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 20 04:59:52.869170 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.869133 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-559bbfdc6f-vjzzs"] Jul 20 04:59:52.894297 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.894034 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sf226"] Jul 20 04:59:52.895417 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.895155 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:52.898254 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.897922 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 20 04:59:52.898254 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.897935 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 20 04:59:52.898254 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.897982 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vkwrd\"" Jul 20 04:59:52.898459 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.898313 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 20 04:59:52.902741 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.902696 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 20 04:59:52.909921 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.909893 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cvb6r"] Jul 20 04:59:52.910049 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.910022 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sf226" Jul 20 04:59:52.912970 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.912924 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bbct5\"" Jul 20 04:59:52.913092 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.912927 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 20 04:59:52.913092 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.912997 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 20 04:59:52.913092 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.912997 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 20 04:59:52.924074 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.924047 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-559bbfdc6f-vjzzs"] Jul 20 04:59:52.924212 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.924085 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sf226"] Jul 20 04:59:52.924212 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.924113 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cvb6r"] Jul 20 04:59:52.924307 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.924211 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:52.926553 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.926533 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 20 04:59:52.926675 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.926664 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 20 04:59:52.926750 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.926713 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jlw9p\"" Jul 20 04:59:52.993893 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:52.993843 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:53.038170 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038136 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.038306 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038173 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-tmp-dir\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:53.038306 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038234 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-config-volume\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:53.038306 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038254 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12256a20-8d87-4866-afcd-51b5ecfb3936-trusted-ca\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.038306 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038279 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk4w4\" (UniqueName: \"kubernetes.io/projected/03c3ada9-2806-4b6e-9e90-ee27aed55433-kube-api-access-gk4w4\") pod \"ingress-canary-sf226\" (UID: \"03c3ada9-2806-4b6e-9e90-ee27aed55433\") " pod="openshift-ingress-canary/ingress-canary-sf226" Jul 20 04:59:53.038435 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038319 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12256a20-8d87-4866-afcd-51b5ecfb3936-ca-trust-extracted\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.038435 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038361 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-certificates\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.038435 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038376 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-bound-sa-token\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.038435 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038390 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03c3ada9-2806-4b6e-9e90-ee27aed55433-cert\") pod \"ingress-canary-sf226\" (UID: \"03c3ada9-2806-4b6e-9e90-ee27aed55433\") " pod="openshift-ingress-canary/ingress-canary-sf226" Jul 20 04:59:53.038435 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038412 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12256a20-8d87-4866-afcd-51b5ecfb3936-installation-pull-secrets\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.038435 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038431 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzs2s\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-kube-api-access-bzs2s\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.038602 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038445 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:53.038602 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038462 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzs4g\" (UniqueName: \"kubernetes.io/projected/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-kube-api-access-pzs4g\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:53.038602 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.038496 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12256a20-8d87-4866-afcd-51b5ecfb3936-image-registry-private-configuration\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.052203 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.052163 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:53.138838 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.138800 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk4w4\" (UniqueName: \"kubernetes.io/projected/03c3ada9-2806-4b6e-9e90-ee27aed55433-kube-api-access-gk4w4\") pod \"ingress-canary-sf226\" (UID: \"03c3ada9-2806-4b6e-9e90-ee27aed55433\") " pod="openshift-ingress-canary/ingress-canary-sf226" Jul 20 04:59:53.138988 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.138842 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12256a20-8d87-4866-afcd-51b5ecfb3936-ca-trust-extracted\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.138988 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.138885 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-certificates\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.138988 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.138904 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-bound-sa-token\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.138988 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.138922 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03c3ada9-2806-4b6e-9e90-ee27aed55433-cert\") pod \"ingress-canary-sf226\" (UID: \"03c3ada9-2806-4b6e-9e90-ee27aed55433\") " pod="openshift-ingress-canary/ingress-canary-sf226" Jul 20 04:59:53.138988 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.138944 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12256a20-8d87-4866-afcd-51b5ecfb3936-installation-pull-secrets\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.138988 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.138988 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzs2s\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-kube-api-access-bzs2s\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.139287 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.139011 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:53.139287 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.139038 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzs4g\" (UniqueName: \"kubernetes.io/projected/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-kube-api-access-pzs4g\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:53.139287 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.139072 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12256a20-8d87-4866-afcd-51b5ecfb3936-image-registry-private-configuration\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.139287 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.139104 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.139287 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.139125 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-tmp-dir\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:53.139287 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.139173 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-config-volume\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:53.139287 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.139204 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12256a20-8d87-4866-afcd-51b5ecfb3936-trusted-ca\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.139287 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:53.139229 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 04:59:53.139287 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:53.139242 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 04:59:53.139287 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:53.139264 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bbfdc6f-vjzzs: secret "image-registry-tls" not found Jul 20 04:59:53.139696 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:53.139313 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls podName:6a13f44c-570d-4fa6-96c6-bcdead54f7ec nodeName:}" failed. No retries permitted until 2026-07-20 04:59:53.639288828 +0000 UTC m=+36.685815712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls") pod "dns-default-cvb6r" (UID: "6a13f44c-570d-4fa6-96c6-bcdead54f7ec") : secret "dns-default-metrics-tls" not found Jul 20 04:59:53.139696 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:53.139334 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls podName:12256a20-8d87-4866-afcd-51b5ecfb3936 nodeName:}" failed. No retries permitted until 2026-07-20 04:59:53.639324131 +0000 UTC m=+36.685851016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls") pod "image-registry-559bbfdc6f-vjzzs" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936") : secret "image-registry-tls" not found Jul 20 04:59:53.139696 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.139504 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-tmp-dir\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:53.139874 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.139822 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-config-volume\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:53.139926 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.139912 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12256a20-8d87-4866-afcd-51b5ecfb3936-ca-trust-extracted\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.140120 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.140100 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-certificates\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.140427 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.140405 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12256a20-8d87-4866-afcd-51b5ecfb3936-trusted-ca\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.143332 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.143303 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12256a20-8d87-4866-afcd-51b5ecfb3936-image-registry-private-configuration\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.143332 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.143308 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03c3ada9-2806-4b6e-9e90-ee27aed55433-cert\") pod \"ingress-canary-sf226\" (UID: \"03c3ada9-2806-4b6e-9e90-ee27aed55433\") " pod="openshift-ingress-canary/ingress-canary-sf226" Jul 20 04:59:53.143492 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.143311 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12256a20-8d87-4866-afcd-51b5ecfb3936-installation-pull-secrets\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.147526 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.147500 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-bound-sa-token\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.147655 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.147637 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzs2s\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-kube-api-access-bzs2s\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.147883 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.147860 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzs4g\" (UniqueName: \"kubernetes.io/projected/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-kube-api-access-pzs4g\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:53.148089 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.148072 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk4w4\" (UniqueName: \"kubernetes.io/projected/03c3ada9-2806-4b6e-9e90-ee27aed55433-kube-api-access-gk4w4\") pod \"ingress-canary-sf226\" (UID: \"03c3ada9-2806-4b6e-9e90-ee27aed55433\") " pod="openshift-ingress-canary/ingress-canary-sf226" Jul 20 04:59:53.221144 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.221061 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sf226" Jul 20 04:59:53.568554 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.568519 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sf226"] Jul 20 04:59:53.572839 ip-10-0-128-33 kubenswrapper[2576]: W0720 04:59:53.572814 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c3ada9_2806_4b6e_9e90_ee27aed55433.slice/crio-e98293ebc061e0bfc073368d24060cbb521de6d4bf798de0195cc62e5164e2a8 WatchSource:0}: Error finding container e98293ebc061e0bfc073368d24060cbb521de6d4bf798de0195cc62e5164e2a8: Status 404 returned error can't find the container with id e98293ebc061e0bfc073368d24060cbb521de6d4bf798de0195cc62e5164e2a8 Jul 20 04:59:53.643482 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.643440 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:53.643482 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.643490 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:53.643678 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:53.643654 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 04:59:53.643757 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:53.643743 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls podName:6a13f44c-570d-4fa6-96c6-bcdead54f7ec nodeName:}" failed. No retries permitted until 2026-07-20 04:59:54.643721521 +0000 UTC m=+37.690248407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls") pod "dns-default-cvb6r" (UID: "6a13f44c-570d-4fa6-96c6-bcdead54f7ec") : secret "dns-default-metrics-tls" not found Jul 20 04:59:53.643806 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:53.643663 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 04:59:53.643806 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:53.643769 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bbfdc6f-vjzzs: secret "image-registry-tls" not found Jul 20 04:59:53.643866 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:53.643834 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls podName:12256a20-8d87-4866-afcd-51b5ecfb3936 nodeName:}" failed. No retries permitted until 2026-07-20 04:59:54.643822541 +0000 UTC m=+37.690349425 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls") pod "image-registry-559bbfdc6f-vjzzs" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936") : secret "image-registry-tls" not found Jul 20 04:59:53.926761 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.926680 2576 generic.go:356] "Generic (PLEG): container finished" podID="f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1" containerID="3000bee61a4917e621a49348d2c1a921709a2e1ff5cf3dcded57b31ec9178154" exitCode=0 Jul 20 04:59:53.927229 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.926768 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhssz" event={"ID":"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1","Type":"ContainerDied","Data":"3000bee61a4917e621a49348d2c1a921709a2e1ff5cf3dcded57b31ec9178154"} Jul 20 04:59:53.927855 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:53.927833 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sf226" event={"ID":"03c3ada9-2806-4b6e-9e90-ee27aed55433","Type":"ContainerStarted","Data":"e98293ebc061e0bfc073368d24060cbb521de6d4bf798de0195cc62e5164e2a8"} Jul 20 04:59:54.647795 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.647762 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 04:59:54.647989 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.647762 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 04:59:54.647989 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.647762 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 04:59:54.650470 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.650449 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z27sm\"" Jul 20 04:59:54.651156 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.651132 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:54.651278 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.651196 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:54.651342 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:54.651287 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 04:59:54.651395 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:54.651352 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls podName:6a13f44c-570d-4fa6-96c6-bcdead54f7ec nodeName:}" failed. No retries permitted until 2026-07-20 04:59:56.651332518 +0000 UTC m=+39.697859405 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls") pod "dns-default-cvb6r" (UID: "6a13f44c-570d-4fa6-96c6-bcdead54f7ec") : secret "dns-default-metrics-tls" not found Jul 20 04:59:54.651395 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:54.651377 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 04:59:54.651395 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:54.651395 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bbfdc6f-vjzzs: secret "image-registry-tls" not found Jul 20 04:59:54.651542 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.651419 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 04:59:54.651542 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:54.651449 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls podName:12256a20-8d87-4866-afcd-51b5ecfb3936 nodeName:}" failed. No retries permitted until 2026-07-20 04:59:56.651431916 +0000 UTC m=+39.697958805 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls") pod "image-registry-559bbfdc6f-vjzzs" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936") : secret "image-registry-tls" not found Jul 20 04:59:54.651542 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.651478 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 04:59:54.651542 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.651478 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 04:59:54.651739 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.651535 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-flt2r\"" Jul 20 04:59:54.651739 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.651573 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 04:59:54.871746 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.871696 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:54.934855 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.934781 2576 generic.go:356] "Generic (PLEG): container finished" podID="f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1" containerID="5e8eebe70fd5b48001f1def1e1bd43d634e967b269965af6588c40fedd9aeca6" exitCode=0 Jul 20 04:59:54.935307 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.934853 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhssz" event={"ID":"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1","Type":"ContainerDied","Data":"5e8eebe70fd5b48001f1def1e1bd43d634e967b269965af6588c40fedd9aeca6"} Jul 20 04:59:54.965162 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:54.965116 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:55.025145 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:55.025109 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:55.886608 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:55.886294 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:55.935583 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:55.935555 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:55.938106 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:55.938083 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sf226" event={"ID":"03c3ada9-2806-4b6e-9e90-ee27aed55433","Type":"ContainerStarted","Data":"cde47f46d70890791106ed6642b726f4e50f056a615f5510359d84bb9d75fa4c"} Jul 20 04:59:55.941284 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:55.941254 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhssz" event={"ID":"f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1","Type":"ContainerStarted","Data":"c811e52b9c0ef9bf3471206b35328bdb68943e5c64e53b570a3ada4051d07819"} Jul 20 04:59:55.953242 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:55.953193 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sf226" podStartSLOduration=1.875860679 podStartE2EDuration="3.953177958s" totalImagesPullingTime="2.077317279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 04:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 04:59:55.952529892 +0000 UTC m=+38.999056851" watchObservedRunningTime="2026-07-20 04:59:55.953177958 +0000 UTC m=+38.999704864" Jul 20 04:59:55.973803 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:55.973752 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lhssz" podStartSLOduration=8.243151978 podStartE2EDuration="37.973733734s" totalImagesPullingTime="29.730581756s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 04:59:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 04:59:55.972487646 +0000 UTC m=+39.019014551" watchObservedRunningTime="2026-07-20 04:59:55.973733734 +0000 UTC m=+39.020260640" Jul 20 04:59:56.099168 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:56.099135 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:56.153864 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:56.153788 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:56.497850 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:56.497769 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:56.547945 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:56.547916 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:56.669143 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:56.669112 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 04:59:56.669248 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:56.669161 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 04:59:56.669288 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:56.669258 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 04:59:56.669288 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:56.669279 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 04:59:56.669346 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:56.669290 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bbfdc6f-vjzzs: secret "image-registry-tls" not found Jul 20 04:59:56.669346 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:56.669315 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls podName:6a13f44c-570d-4fa6-96c6-bcdead54f7ec nodeName:}" failed. No retries permitted until 2026-07-20 05:00:00.669301149 +0000 UTC m=+43.715828033 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls") pod "dns-default-cvb6r" (UID: "6a13f44c-570d-4fa6-96c6-bcdead54f7ec") : secret "dns-default-metrics-tls" not found Jul 20 04:59:56.669346 ip-10-0-128-33 kubenswrapper[2576]: E0720 04:59:56.669329 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls podName:12256a20-8d87-4866-afcd-51b5ecfb3936 nodeName:}" failed. No retries permitted until 2026-07-20 05:00:00.669323375 +0000 UTC m=+43.715850259 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls") pod "image-registry-559bbfdc6f-vjzzs" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936") : secret "image-registry-tls" not found Jul 20 04:59:57.288832 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:57.288798 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:57.340793 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:57.340761 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 04:59:58.828283 ip-10-0-128-33 kubenswrapper[2576]: I0720 04:59:58.828251 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:00:00.698721 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:00.698676 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 05:00:00.699171 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:00.698757 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 05:00:00.699171 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:00.698833 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 05:00:00.699171 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:00.698856 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bbfdc6f-vjzzs: secret "image-registry-tls" not found Jul 20 05:00:00.699171 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:00.698906 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls podName:12256a20-8d87-4866-afcd-51b5ecfb3936 nodeName:}" failed. No retries permitted until 2026-07-20 05:00:08.698892107 +0000 UTC m=+51.745418992 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls") pod "image-registry-559bbfdc6f-vjzzs" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936") : secret "image-registry-tls" not found Jul 20 05:00:00.699171 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:00.698846 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 05:00:00.699171 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:00.699001 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls podName:6a13f44c-570d-4fa6-96c6-bcdead54f7ec nodeName:}" failed. No retries permitted until 2026-07-20 05:00:08.698946381 +0000 UTC m=+51.745473265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls") pod "dns-default-cvb6r" (UID: "6a13f44c-570d-4fa6-96c6-bcdead54f7ec") : secret "dns-default-metrics-tls" not found Jul 20 05:00:06.657198 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:06.657167 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:00:06.714623 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:06.714587 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:00:08.002668 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:08.002635 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:00:08.105241 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:08.105210 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:00:08.758874 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:08.758830 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 05:00:08.758874 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:08.758885 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 05:00:08.759142 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:08.758937 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 05:00:08.759142 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:08.758990 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 05:00:08.759142 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:08.759000 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bbfdc6f-vjzzs: secret "image-registry-tls" not found Jul 20 05:00:08.759142 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:08.759013 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls podName:6a13f44c-570d-4fa6-96c6-bcdead54f7ec nodeName:}" failed. No retries permitted until 2026-07-20 05:00:24.758996955 +0000 UTC m=+67.805523839 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls") pod "dns-default-cvb6r" (UID: "6a13f44c-570d-4fa6-96c6-bcdead54f7ec") : secret "dns-default-metrics-tls" not found Jul 20 05:00:08.759142 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:08.759037 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls podName:12256a20-8d87-4866-afcd-51b5ecfb3936 nodeName:}" failed. No retries permitted until 2026-07-20 05:00:24.759025613 +0000 UTC m=+67.805552497 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls") pod "image-registry-559bbfdc6f-vjzzs" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936") : secret "image-registry-tls" not found Jul 20 05:00:10.966810 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:10.966779 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:00:11.017214 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:11.017180 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:00:18.922427 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:18.922398 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-85lxq" Jul 20 05:00:19.069265 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:19.069232 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:00:19.169459 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:19.169432 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:00:23.361408 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.361368 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 05:00:23.364034 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.364015 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 05:00:23.371914 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:23.371897 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 20 05:00:23.371973 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:23.371966 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs podName:06a59c1d-4c72-423f-9e8d-27db0a919dba nodeName:}" failed. No retries permitted until 2026-07-20 05:01:27.371938923 +0000 UTC m=+130.418465806 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs") pod "network-metrics-daemon-gnmjm" (UID: "06a59c1d-4c72-423f-9e8d-27db0a919dba") : secret "metrics-daemon-secret" not found Jul 20 05:00:23.462727 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.462688 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 05:00:23.465450 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.465428 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 05:00:23.477728 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.477695 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9dd925c-ebe0-4584-a59a-bcc23cb8a69b-original-pull-secret\") pod \"global-pull-secret-syncer-dh2cl\" (UID: \"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b\") " pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 05:00:23.563684 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.563638 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl7dt\" (UniqueName: \"kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt\") pod \"network-check-target-nwhn4\" (UID: \"1d8591ce-a880-4fa7-9792-313af6b166e0\") " pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 05:00:23.566071 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.566052 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 05:00:23.576006 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.575984 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 05:00:23.587157 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.587126 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl7dt\" (UniqueName: \"kubernetes.io/projected/1d8591ce-a880-4fa7-9792-313af6b166e0-kube-api-access-rl7dt\") pod \"network-check-target-nwhn4\" (UID: \"1d8591ce-a880-4fa7-9792-313af6b166e0\") " pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 05:00:23.769209 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.769180 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-flt2r\"" Jul 20 05:00:23.772202 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.772185 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dh2cl" Jul 20 05:00:23.777333 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.777311 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 05:00:23.910378 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.910348 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dh2cl"] Jul 20 05:00:23.913168 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:00:23.913143 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9dd925c_ebe0_4584_a59a_bcc23cb8a69b.slice/crio-377fa1d5a4103620537999231cd0e18da7c887b1225609973d6fe0ff7028453e WatchSource:0}: Error finding container 377fa1d5a4103620537999231cd0e18da7c887b1225609973d6fe0ff7028453e: Status 404 returned error can't find the container with id 377fa1d5a4103620537999231cd0e18da7c887b1225609973d6fe0ff7028453e Jul 20 05:00:23.925041 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:23.925015 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nwhn4"] Jul 20 05:00:23.928654 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:00:23.928629 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d8591ce_a880_4fa7_9792_313af6b166e0.slice/crio-b1e1dda9cedd00802835d59f553782ad55bf934f15692fa2705d8162a4bf6cd2 WatchSource:0}: Error finding container b1e1dda9cedd00802835d59f553782ad55bf934f15692fa2705d8162a4bf6cd2: Status 404 returned error can't find the container with id b1e1dda9cedd00802835d59f553782ad55bf934f15692fa2705d8162a4bf6cd2 Jul 20 05:00:24.006911 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:24.006876 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dh2cl" event={"ID":"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b","Type":"ContainerStarted","Data":"377fa1d5a4103620537999231cd0e18da7c887b1225609973d6fe0ff7028453e"} Jul 20 05:00:24.007865 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:24.007840 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nwhn4" event={"ID":"1d8591ce-a880-4fa7-9792-313af6b166e0","Type":"ContainerStarted","Data":"b1e1dda9cedd00802835d59f553782ad55bf934f15692fa2705d8162a4bf6cd2"} Jul 20 05:00:24.772653 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:24.772613 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 05:00:24.772653 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:24.772671 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 05:00:24.773327 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:24.772803 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 05:00:24.773327 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:24.772821 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bbfdc6f-vjzzs: secret "image-registry-tls" not found Jul 20 05:00:24.773327 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:24.772871 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls podName:12256a20-8d87-4866-afcd-51b5ecfb3936 nodeName:}" failed. No retries permitted until 2026-07-20 05:00:56.772854739 +0000 UTC m=+99.819381624 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls") pod "image-registry-559bbfdc6f-vjzzs" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936") : secret "image-registry-tls" not found Jul 20 05:00:24.773327 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:24.773052 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 05:00:24.773327 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:24.773124 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls podName:6a13f44c-570d-4fa6-96c6-bcdead54f7ec nodeName:}" failed. No retries permitted until 2026-07-20 05:00:56.773105029 +0000 UTC m=+99.819631914 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls") pod "dns-default-cvb6r" (UID: "6a13f44c-570d-4fa6-96c6-bcdead54f7ec") : secret "dns-default-metrics-tls" not found Jul 20 05:00:30.025803 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:30.025762 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dh2cl" event={"ID":"a9dd925c-ebe0-4584-a59a-bcc23cb8a69b","Type":"ContainerStarted","Data":"18f167f5880fef82f7816bdc6022c77034031fa9755b7c08b2c65740b4bfc28e"} Jul 20 05:00:30.027185 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:30.027162 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nwhn4" event={"ID":"1d8591ce-a880-4fa7-9792-313af6b166e0","Type":"ContainerStarted","Data":"c97103e386bd9791e0075b5468769d602a28d5bbd69135c38e02a77abf68b013"} Jul 20 05:00:30.027310 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:30.027292 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 05:00:30.041362 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:30.041311 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dh2cl" podStartSLOduration=67.021214242 podStartE2EDuration="1m12.041297312s" totalImagesPullingTime="5.02008307s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:00:30.040770142 +0000 UTC m=+73.087297070" watchObservedRunningTime="2026-07-20 05:00:30.041297312 +0000 UTC m=+73.087824219" Jul 20 05:00:30.055014 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:30.054935 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nwhn4" podStartSLOduration=67.063611323 podStartE2EDuration="1m12.054913682s" totalImagesPullingTime="4.991302359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:00:30.054839132 +0000 UTC m=+73.101366038" watchObservedRunningTime="2026-07-20 05:00:30.054913682 +0000 UTC m=+73.101440589" Jul 20 05:00:32.910966 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:32.910932 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:00:33.010896 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:33.010864 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:00:33.060518 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:33.060485 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:00:56.798032 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:56.797990 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 05:00:56.798450 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:00:56.798046 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 05:00:56.798450 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:56.798133 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 05:00:56.798450 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:56.798140 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 05:00:56.798450 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:56.798160 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bbfdc6f-vjzzs: secret "image-registry-tls" not found Jul 20 05:00:56.798450 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:56.798206 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls podName:6a13f44c-570d-4fa6-96c6-bcdead54f7ec nodeName:}" failed. No retries permitted until 2026-07-20 05:02:00.798188893 +0000 UTC m=+163.844715782 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls") pod "dns-default-cvb6r" (UID: "6a13f44c-570d-4fa6-96c6-bcdead54f7ec") : secret "dns-default-metrics-tls" not found Jul 20 05:00:56.798450 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:00:56.798222 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls podName:12256a20-8d87-4866-afcd-51b5ecfb3936 nodeName:}" failed. No retries permitted until 2026-07-20 05:02:00.798215305 +0000 UTC m=+163.844742189 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls") pod "image-registry-559bbfdc6f-vjzzs" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936") : secret "image-registry-tls" not found Jul 20 05:01:01.032804 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:01:01.032766 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nwhn4" Jul 20 05:01:07.731115 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:01:07.731084 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:01:07.781997 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:01:07.781944 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:01:27.409240 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:01:27.409185 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 05:01:27.409731 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:01:27.409354 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 20 05:01:27.409731 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:01:27.409420 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs podName:06a59c1d-4c72-423f-9e8d-27db0a919dba nodeName:}" failed. No retries permitted until 2026-07-20 05:03:29.409404813 +0000 UTC m=+252.455931700 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs") pod "network-metrics-daemon-gnmjm" (UID: "06a59c1d-4c72-423f-9e8d-27db0a919dba") : secret "metrics-daemon-secret" not found Jul 20 05:01:55.906915 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:01:55.906872 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" podUID="12256a20-8d87-4866-afcd-51b5ecfb3936" Jul 20 05:01:55.937618 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:01:55.937570 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-cvb6r" podUID="6a13f44c-570d-4fa6-96c6-bcdead54f7ec" Jul 20 05:01:56.231759 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:01:56.231738 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 05:01:57.659543 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:01:57.659501 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 05:02:00.844975 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:02:00.844901 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls\") pod \"image-registry-559bbfdc6f-vjzzs\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 05:02:00.845472 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:02:00.845068 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 05:02:00.845472 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:02:00.845081 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 05:02:00.845472 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:02:00.845106 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559bbfdc6f-vjzzs: secret "image-registry-tls" not found Jul 20 05:02:00.845472 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:02:00.845152 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 05:02:00.845472 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:02:00.845180 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls podName:12256a20-8d87-4866-afcd-51b5ecfb3936 nodeName:}" failed. No retries permitted until 2026-07-20 05:04:02.845159676 +0000 UTC m=+285.891686576 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls") pod "image-registry-559bbfdc6f-vjzzs" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936") : secret "image-registry-tls" not found Jul 20 05:02:00.845472 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:02:00.845200 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls podName:6a13f44c-570d-4fa6-96c6-bcdead54f7ec nodeName:}" failed. No retries permitted until 2026-07-20 05:04:02.845189443 +0000 UTC m=+285.891716330 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls") pod "dns-default-cvb6r" (UID: "6a13f44c-570d-4fa6-96c6-bcdead54f7ec") : secret "dns-default-metrics-tls" not found Jul 20 05:02:08.647424 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:02:08.647381 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cvb6r" Jul 20 05:02:11.647529 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:02:11.647445 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 05:03:25.605099 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.605061 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b"] Jul 20 05:03:25.607882 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.607862 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" Jul 20 05:03:25.611809 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.611782 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 20 05:03:25.612074 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.612050 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 20 05:03:25.612539 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.612515 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 20 05:03:25.612669 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.612619 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 20 05:03:25.612989 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.612809 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wkrv6\"" Jul 20 05:03:25.615839 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.615766 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-c9l2r"] Jul 20 05:03:25.620110 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.620089 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-q8k7n"] Jul 20 05:03:25.620228 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.620205 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.622746 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.622723 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9mdfj"] Jul 20 05:03:25.622860 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.622813 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.623367 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.623124 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 20 05:03:25.623367 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.623232 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 20 05:03:25.623367 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.623260 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 20 05:03:25.623367 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.623280 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xpmlj\"" Jul 20 05:03:25.623367 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.623284 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 20 05:03:25.626052 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.626034 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-64bbb4d846-ftbct"] Jul 20 05:03:25.626186 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.626161 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9mdfj" Jul 20 05:03:25.626989 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.626939 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 20 05:03:25.627086 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.626972 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 20 05:03:25.627169 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.627150 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 20 05:03:25.627426 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.627258 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ll5l4\"" Jul 20 05:03:25.628427 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.628411 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 20 05:03:25.629146 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.629123 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 20 05:03:25.629352 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.629331 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b"] Jul 20 05:03:25.629457 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.629438 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.629710 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.629691 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 20 05:03:25.630278 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.630261 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2k7fs\"" Jul 20 05:03:25.630973 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.630935 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9mdfj"] Jul 20 05:03:25.632997 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.632980 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 20 05:03:25.633679 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.633502 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 20 05:03:25.634159 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.634013 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rnk54\"" Jul 20 05:03:25.634159 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.634069 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 20 05:03:25.634586 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.634017 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 20 05:03:25.634586 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.634495 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-c9l2r"] Jul 20 05:03:25.634741 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.634726 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 20 05:03:25.635179 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.635156 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-q8k7n"] Jul 20 05:03:25.635257 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.635245 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 20 05:03:25.635371 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.635248 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 20 05:03:25.635671 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.635648 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 20 05:03:25.637079 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.637061 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64bbb4d846-ftbct"] Jul 20 05:03:25.658625 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.658600 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.658779 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.658636 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-default-certificate\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.658779 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.658665 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dced5a3f-757b-47aa-b3b2-2ada2072603f-serving-cert\") pod \"service-ca-operator-5bcbc94c78-48w7b\" (UID: \"dced5a3f-757b-47aa-b3b2-2ada2072603f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" Jul 20 05:03:25.658779 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.658690 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/240fe5fe-623e-4495-a594-63f72c28f6bd-trusted-ca\") pod \"console-operator-565b7b569d-q8k7n\" (UID: \"240fe5fe-623e-4495-a594-63f72c28f6bd\") " pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.658779 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.658714 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-288mj\" (UniqueName: \"kubernetes.io/projected/9661ba55-2be6-499e-a1af-a5c0cb54f376-kube-api-access-288mj\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.658779 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.658742 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/240fe5fe-623e-4495-a594-63f72c28f6bd-serving-cert\") pod \"console-operator-565b7b569d-q8k7n\" (UID: \"240fe5fe-623e-4495-a594-63f72c28f6bd\") " pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.659131 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.658781 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cq42\" (UniqueName: \"kubernetes.io/projected/8bf73698-17d9-4e7f-b633-d7d07e7df680-kube-api-access-8cq42\") pod \"volume-data-source-validator-78cf4c6f6-9mdfj\" (UID: \"8bf73698-17d9-4e7f-b633-d7d07e7df680\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9mdfj" Jul 20 05:03:25.659131 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.658813 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dced5a3f-757b-47aa-b3b2-2ada2072603f-config\") pod \"service-ca-operator-5bcbc94c78-48w7b\" (UID: \"dced5a3f-757b-47aa-b3b2-2ada2072603f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" Jul 20 05:03:25.659131 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.658844 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9661ba55-2be6-499e-a1af-a5c0cb54f376-service-ca-bundle\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.659131 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.658871 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.659131 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.658895 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76hwj\" (UniqueName: \"kubernetes.io/projected/1877c3b7-fa1e-49c5-a296-ef9478df42e8-kube-api-access-76hwj\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.659131 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.658945 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9661ba55-2be6-499e-a1af-a5c0cb54f376-tmp\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.659131 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.658991 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9661ba55-2be6-499e-a1af-a5c0cb54f376-serving-cert\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.659131 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.659011 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/240fe5fe-623e-4495-a594-63f72c28f6bd-config\") pod \"console-operator-565b7b569d-q8k7n\" (UID: \"240fe5fe-623e-4495-a594-63f72c28f6bd\") " pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.659131 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.659029 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbgrw\" (UniqueName: \"kubernetes.io/projected/dced5a3f-757b-47aa-b3b2-2ada2072603f-kube-api-access-sbgrw\") pod \"service-ca-operator-5bcbc94c78-48w7b\" (UID: \"dced5a3f-757b-47aa-b3b2-2ada2072603f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" Jul 20 05:03:25.659131 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.659109 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9661ba55-2be6-499e-a1af-a5c0cb54f376-snapshots\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.659131 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.659127 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b8xx\" (UniqueName: \"kubernetes.io/projected/240fe5fe-623e-4495-a594-63f72c28f6bd-kube-api-access-5b8xx\") pod \"console-operator-565b7b569d-q8k7n\" (UID: \"240fe5fe-623e-4495-a594-63f72c28f6bd\") " pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.659511 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.659172 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-stats-auth\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.659511 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.659210 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9661ba55-2be6-499e-a1af-a5c0cb54f376-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.705590 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.705555 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv"] Jul 20 05:03:25.708669 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.708651 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c"] Jul 20 05:03:25.708783 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.708762 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" Jul 20 05:03:25.711410 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.711388 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 20 05:03:25.711517 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.711389 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ls977\"" Jul 20 05:03:25.711558 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.711540 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 20 05:03:25.711691 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.711675 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 20 05:03:25.712303 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.712283 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:25.714331 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.714304 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 20 05:03:25.714546 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.714407 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 20 05:03:25.714546 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.714437 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 20 05:03:25.715816 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.715704 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 20 05:03:25.716319 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.716297 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-g5tb8\"" Jul 20 05:03:25.720748 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.720724 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv"] Jul 20 05:03:25.721071 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.721047 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c"] Jul 20 05:03:25.760414 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.760382 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b8xx\" (UniqueName: \"kubernetes.io/projected/240fe5fe-623e-4495-a594-63f72c28f6bd-kube-api-access-5b8xx\") pod \"console-operator-565b7b569d-q8k7n\" (UID: \"240fe5fe-623e-4495-a594-63f72c28f6bd\") " pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.760414 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.760426 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-kndfv\" (UID: \"7f26fac3-f0a1-4461-8600-bf0a805d6cc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" Jul 20 05:03:25.760706 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.760459 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-stats-auth\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.760706 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.760494 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9661ba55-2be6-499e-a1af-a5c0cb54f376-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.760706 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.760518 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6qqk\" (UniqueName: \"kubernetes.io/projected/4d64666c-dff4-424e-a774-9f90c94a206c-kube-api-access-r6qqk\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:25.760706 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.760543 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.760706 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.760571 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-default-certificate\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.760706 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.760692 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dced5a3f-757b-47aa-b3b2-2ada2072603f-serving-cert\") pod \"service-ca-operator-5bcbc94c78-48w7b\" (UID: \"dced5a3f-757b-47aa-b3b2-2ada2072603f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" Jul 20 05:03:25.761122 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:25.760717 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle podName:1877c3b7-fa1e-49c5-a296-ef9478df42e8 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:26.260698645 +0000 UTC m=+249.307225530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle") pod "router-default-64bbb4d846-ftbct" (UID: "1877c3b7-fa1e-49c5-a296-ef9478df42e8") : configmap references non-existent config key: service-ca.crt Jul 20 05:03:25.761122 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.760758 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/240fe5fe-623e-4495-a594-63f72c28f6bd-trusted-ca\") pod \"console-operator-565b7b569d-q8k7n\" (UID: \"240fe5fe-623e-4495-a594-63f72c28f6bd\") " pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.761122 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.760786 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288mj\" (UniqueName: \"kubernetes.io/projected/9661ba55-2be6-499e-a1af-a5c0cb54f376-kube-api-access-288mj\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.761122 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.760811 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/240fe5fe-623e-4495-a594-63f72c28f6bd-serving-cert\") pod \"console-operator-565b7b569d-q8k7n\" (UID: \"240fe5fe-623e-4495-a594-63f72c28f6bd\") " pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.761122 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.760849 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cq42\" (UniqueName: \"kubernetes.io/projected/8bf73698-17d9-4e7f-b633-d7d07e7df680-kube-api-access-8cq42\") pod \"volume-data-source-validator-78cf4c6f6-9mdfj\" (UID: \"8bf73698-17d9-4e7f-b633-d7d07e7df680\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9mdfj" Jul 20 05:03:25.761122 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.760880 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dced5a3f-757b-47aa-b3b2-2ada2072603f-config\") pod \"service-ca-operator-5bcbc94c78-48w7b\" (UID: \"dced5a3f-757b-47aa-b3b2-2ada2072603f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" Jul 20 05:03:25.761421 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.761364 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9661ba55-2be6-499e-a1af-a5c0cb54f376-service-ca-bundle\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.761421 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.761400 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.761531 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.761427 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76hwj\" (UniqueName: \"kubernetes.io/projected/1877c3b7-fa1e-49c5-a296-ef9478df42e8-kube-api-access-76hwj\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.761531 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.761468 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9661ba55-2be6-499e-a1af-a5c0cb54f376-tmp\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.761531 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.761496 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9661ba55-2be6-499e-a1af-a5c0cb54f376-serving-cert\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.761531 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.761525 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:25.761735 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.761556 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/240fe5fe-623e-4495-a594-63f72c28f6bd-config\") pod \"console-operator-565b7b569d-q8k7n\" (UID: \"240fe5fe-623e-4495-a594-63f72c28f6bd\") " pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.761735 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.761587 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbgrw\" (UniqueName: \"kubernetes.io/projected/dced5a3f-757b-47aa-b3b2-2ada2072603f-kube-api-access-sbgrw\") pod \"service-ca-operator-5bcbc94c78-48w7b\" (UID: \"dced5a3f-757b-47aa-b3b2-2ada2072603f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" Jul 20 05:03:25.761735 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.761589 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9661ba55-2be6-499e-a1af-a5c0cb54f376-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.761735 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.761679 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9661ba55-2be6-499e-a1af-a5c0cb54f376-snapshots\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.761735 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.761710 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4d64666c-dff4-424e-a774-9f90c94a206c-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:25.761987 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.761739 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrr9g\" (UniqueName: \"kubernetes.io/projected/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-kube-api-access-lrr9g\") pod \"cluster-samples-operator-79b7c55f87-kndfv\" (UID: \"7f26fac3-f0a1-4461-8600-bf0a805d6cc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" Jul 20 05:03:25.762304 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.762280 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9661ba55-2be6-499e-a1af-a5c0cb54f376-service-ca-bundle\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.762385 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.762344 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dced5a3f-757b-47aa-b3b2-2ada2072603f-config\") pod \"service-ca-operator-5bcbc94c78-48w7b\" (UID: \"dced5a3f-757b-47aa-b3b2-2ada2072603f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" Jul 20 05:03:25.762385 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:25.762375 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 05:03:25.762496 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.762382 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9661ba55-2be6-499e-a1af-a5c0cb54f376-snapshots\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.762496 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:25.762423 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs podName:1877c3b7-fa1e-49c5-a296-ef9478df42e8 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:26.26240596 +0000 UTC m=+249.308932857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs") pod "router-default-64bbb4d846-ftbct" (UID: "1877c3b7-fa1e-49c5-a296-ef9478df42e8") : secret "router-metrics-certs-default" not found Jul 20 05:03:25.762735 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.762714 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/240fe5fe-623e-4495-a594-63f72c28f6bd-trusted-ca\") pod \"console-operator-565b7b569d-q8k7n\" (UID: \"240fe5fe-623e-4495-a594-63f72c28f6bd\") " pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.762923 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.762899 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9661ba55-2be6-499e-a1af-a5c0cb54f376-tmp\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.763130 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.763086 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/240fe5fe-623e-4495-a594-63f72c28f6bd-config\") pod \"console-operator-565b7b569d-q8k7n\" (UID: \"240fe5fe-623e-4495-a594-63f72c28f6bd\") " pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.763504 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.763479 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dced5a3f-757b-47aa-b3b2-2ada2072603f-serving-cert\") pod \"service-ca-operator-5bcbc94c78-48w7b\" (UID: \"dced5a3f-757b-47aa-b3b2-2ada2072603f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" Jul 20 05:03:25.763582 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.763499 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-stats-auth\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.763582 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.763545 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/240fe5fe-623e-4495-a594-63f72c28f6bd-serving-cert\") pod \"console-operator-565b7b569d-q8k7n\" (UID: \"240fe5fe-623e-4495-a594-63f72c28f6bd\") " pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.763777 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.763723 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-default-certificate\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.764481 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.764464 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9661ba55-2be6-499e-a1af-a5c0cb54f376-serving-cert\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.768945 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.768920 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-288mj\" (UniqueName: \"kubernetes.io/projected/9661ba55-2be6-499e-a1af-a5c0cb54f376-kube-api-access-288mj\") pod \"insights-operator-555d9d7885-c9l2r\" (UID: \"9661ba55-2be6-499e-a1af-a5c0cb54f376\") " pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.769369 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.769341 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b8xx\" (UniqueName: \"kubernetes.io/projected/240fe5fe-623e-4495-a594-63f72c28f6bd-kube-api-access-5b8xx\") pod \"console-operator-565b7b569d-q8k7n\" (UID: \"240fe5fe-623e-4495-a594-63f72c28f6bd\") " pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.769555 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.769539 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cq42\" (UniqueName: \"kubernetes.io/projected/8bf73698-17d9-4e7f-b633-d7d07e7df680-kube-api-access-8cq42\") pod \"volume-data-source-validator-78cf4c6f6-9mdfj\" (UID: \"8bf73698-17d9-4e7f-b633-d7d07e7df680\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9mdfj" Jul 20 05:03:25.769781 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.769750 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76hwj\" (UniqueName: \"kubernetes.io/projected/1877c3b7-fa1e-49c5-a296-ef9478df42e8-kube-api-access-76hwj\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:25.770273 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.770253 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbgrw\" (UniqueName: \"kubernetes.io/projected/dced5a3f-757b-47aa-b3b2-2ada2072603f-kube-api-access-sbgrw\") pod \"service-ca-operator-5bcbc94c78-48w7b\" (UID: \"dced5a3f-757b-47aa-b3b2-2ada2072603f\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" Jul 20 05:03:25.809975 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.809937 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:25.860770 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.860686 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:25.862767 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.862749 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6qqk\" (UniqueName: \"kubernetes.io/projected/4d64666c-dff4-424e-a774-9f90c94a206c-kube-api-access-r6qqk\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:25.862840 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.862827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:25.862881 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.862870 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4d64666c-dff4-424e-a774-9f90c94a206c-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:25.862921 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.862886 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrr9g\" (UniqueName: \"kubernetes.io/projected/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-kube-api-access-lrr9g\") pod \"cluster-samples-operator-79b7c55f87-kndfv\" (UID: \"7f26fac3-f0a1-4461-8600-bf0a805d6cc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" Jul 20 05:03:25.862921 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.862903 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-kndfv\" (UID: \"7f26fac3-f0a1-4461-8600-bf0a805d6cc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" Jul 20 05:03:25.863036 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:25.862981 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 05:03:25.863036 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:25.863003 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 05:03:25.863128 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:25.863075 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls podName:7f26fac3-f0a1-4461-8600-bf0a805d6cc6 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:26.363050031 +0000 UTC m=+249.409576933 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-kndfv" (UID: "7f26fac3-f0a1-4461-8600-bf0a805d6cc6") : secret "samples-operator-tls" not found Jul 20 05:03:25.863128 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:25.863089 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls podName:4d64666c-dff4-424e-a774-9f90c94a206c nodeName:}" failed. No retries permitted until 2026-07-20 05:03:26.363082469 +0000 UTC m=+249.409609353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-td56c" (UID: "4d64666c-dff4-424e-a774-9f90c94a206c") : secret "cluster-monitoring-operator-tls" not found Jul 20 05:03:25.863603 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.863579 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4d64666c-dff4-424e-a774-9f90c94a206c-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:25.871071 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.871046 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrr9g\" (UniqueName: \"kubernetes.io/projected/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-kube-api-access-lrr9g\") pod \"cluster-samples-operator-79b7c55f87-kndfv\" (UID: \"7f26fac3-f0a1-4461-8600-bf0a805d6cc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" Jul 20 05:03:25.871168 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.871046 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6qqk\" (UniqueName: \"kubernetes.io/projected/4d64666c-dff4-424e-a774-9f90c94a206c-kube-api-access-r6qqk\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:25.920061 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.920027 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" Jul 20 05:03:25.932454 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.932431 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-c9l2r" Jul 20 05:03:25.940343 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.940316 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:25.946370 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:25.946342 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9mdfj" Jul 20 05:03:26.077284 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:26.077123 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b"] Jul 20 05:03:26.081031 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:03:26.081002 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddced5a3f_757b_47aa_b3b2_2ada2072603f.slice/crio-4b38919ce9e7f896408372d5e5ff148e7b78536555bb38523c7bd35320f31f05 WatchSource:0}: Error finding container 4b38919ce9e7f896408372d5e5ff148e7b78536555bb38523c7bd35320f31f05: Status 404 returned error can't find the container with id 4b38919ce9e7f896408372d5e5ff148e7b78536555bb38523c7bd35320f31f05 Jul 20 05:03:26.123195 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:26.123103 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9mdfj"] Jul 20 05:03:26.127034 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:03:26.127007 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf73698_17d9_4e7f_b633_d7d07e7df680.slice/crio-243206759f06436176cae6e4171c9fe3ab1b7c03e282eb86cc7873e3b86eaf21 WatchSource:0}: Error finding container 243206759f06436176cae6e4171c9fe3ab1b7c03e282eb86cc7873e3b86eaf21: Status 404 returned error can't find the container with id 243206759f06436176cae6e4171c9fe3ab1b7c03e282eb86cc7873e3b86eaf21 Jul 20 05:03:26.266979 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:26.266918 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:26.267157 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:26.267006 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:26.267157 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:26.267089 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle podName:1877c3b7-fa1e-49c5-a296-ef9478df42e8 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:27.267070636 +0000 UTC m=+250.313597519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle") pod "router-default-64bbb4d846-ftbct" (UID: "1877c3b7-fa1e-49c5-a296-ef9478df42e8") : configmap references non-existent config key: service-ca.crt Jul 20 05:03:26.267157 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:26.267127 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 05:03:26.267265 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:26.267176 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs podName:1877c3b7-fa1e-49c5-a296-ef9478df42e8 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:27.267164978 +0000 UTC m=+250.313691863 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs") pod "router-default-64bbb4d846-ftbct" (UID: "1877c3b7-fa1e-49c5-a296-ef9478df42e8") : secret "router-metrics-certs-default" not found Jul 20 05:03:26.302934 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:26.302902 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-q8k7n"] Jul 20 05:03:26.306271 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:03:26.306239 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod240fe5fe_623e_4495_a594_63f72c28f6bd.slice/crio-2c0849070a766845168fb882f451e7f182adb221fe9a86f7b6201584848dc886 WatchSource:0}: Error finding container 2c0849070a766845168fb882f451e7f182adb221fe9a86f7b6201584848dc886: Status 404 returned error can't find the container with id 2c0849070a766845168fb882f451e7f182adb221fe9a86f7b6201584848dc886 Jul 20 05:03:26.306465 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:26.306441 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-c9l2r"] Jul 20 05:03:26.309355 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:03:26.309331 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9661ba55_2be6_499e_a1af_a5c0cb54f376.slice/crio-f7749c98d829c419387a623c00e96b7a86db83a71604d7d7d00fab69f052e7cd WatchSource:0}: Error finding container f7749c98d829c419387a623c00e96b7a86db83a71604d7d7d00fab69f052e7cd: Status 404 returned error can't find the container with id f7749c98d829c419387a623c00e96b7a86db83a71604d7d7d00fab69f052e7cd Jul 20 05:03:26.368159 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:26.368115 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-kndfv\" (UID: \"7f26fac3-f0a1-4461-8600-bf0a805d6cc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" Jul 20 05:03:26.368324 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:26.368215 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:26.368324 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:26.368273 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 05:03:26.368324 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:26.368298 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 05:03:26.368423 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:26.368343 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls podName:7f26fac3-f0a1-4461-8600-bf0a805d6cc6 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:27.368326127 +0000 UTC m=+250.414853012 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-kndfv" (UID: "7f26fac3-f0a1-4461-8600-bf0a805d6cc6") : secret "samples-operator-tls" not found Jul 20 05:03:26.368423 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:26.368362 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls podName:4d64666c-dff4-424e-a774-9f90c94a206c nodeName:}" failed. No retries permitted until 2026-07-20 05:03:27.368353187 +0000 UTC m=+250.414880071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-td56c" (UID: "4d64666c-dff4-424e-a774-9f90c94a206c") : secret "cluster-monitoring-operator-tls" not found Jul 20 05:03:26.448487 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:26.448391 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" event={"ID":"240fe5fe-623e-4495-a594-63f72c28f6bd","Type":"ContainerStarted","Data":"2c0849070a766845168fb882f451e7f182adb221fe9a86f7b6201584848dc886"} Jul 20 05:03:26.449404 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:26.449379 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-c9l2r" event={"ID":"9661ba55-2be6-499e-a1af-a5c0cb54f376","Type":"ContainerStarted","Data":"f7749c98d829c419387a623c00e96b7a86db83a71604d7d7d00fab69f052e7cd"} Jul 20 05:03:26.450437 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:26.450412 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" event={"ID":"dced5a3f-757b-47aa-b3b2-2ada2072603f","Type":"ContainerStarted","Data":"4b38919ce9e7f896408372d5e5ff148e7b78536555bb38523c7bd35320f31f05"} Jul 20 05:03:26.451431 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:26.451411 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9mdfj" event={"ID":"8bf73698-17d9-4e7f-b633-d7d07e7df680","Type":"ContainerStarted","Data":"243206759f06436176cae6e4171c9fe3ab1b7c03e282eb86cc7873e3b86eaf21"} Jul 20 05:03:27.277212 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:27.276307 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:27.277212 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:27.276407 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:27.277212 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:27.276641 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 05:03:27.277212 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:27.276710 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs podName:1877c3b7-fa1e-49c5-a296-ef9478df42e8 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:29.276690457 +0000 UTC m=+252.323217343 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs") pod "router-default-64bbb4d846-ftbct" (UID: "1877c3b7-fa1e-49c5-a296-ef9478df42e8") : secret "router-metrics-certs-default" not found Jul 20 05:03:27.277212 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:27.277171 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle podName:1877c3b7-fa1e-49c5-a296-ef9478df42e8 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:29.277151969 +0000 UTC m=+252.323678854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle") pod "router-default-64bbb4d846-ftbct" (UID: "1877c3b7-fa1e-49c5-a296-ef9478df42e8") : configmap references non-existent config key: service-ca.crt Jul 20 05:03:27.377213 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:27.377175 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:27.377400 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:27.377258 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-kndfv\" (UID: \"7f26fac3-f0a1-4461-8600-bf0a805d6cc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" Jul 20 05:03:27.377400 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:27.377387 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 05:03:27.377515 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:27.377451 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls podName:7f26fac3-f0a1-4461-8600-bf0a805d6cc6 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:29.377431474 +0000 UTC m=+252.423958365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-kndfv" (UID: "7f26fac3-f0a1-4461-8600-bf0a805d6cc6") : secret "samples-operator-tls" not found Jul 20 05:03:27.377886 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:27.377849 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 05:03:27.378014 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:27.377929 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls podName:4d64666c-dff4-424e-a774-9f90c94a206c nodeName:}" failed. No retries permitted until 2026-07-20 05:03:29.377912691 +0000 UTC m=+252.424439581 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-td56c" (UID: "4d64666c-dff4-424e-a774-9f90c94a206c") : secret "cluster-monitoring-operator-tls" not found Jul 20 05:03:28.459455 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:28.459414 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9mdfj" event={"ID":"8bf73698-17d9-4e7f-b633-d7d07e7df680","Type":"ContainerStarted","Data":"ba9f55606381c406cd994761d03af1dec51f492a0eea4dff99f032e0f1e6d3ed"} Jul 20 05:03:28.473672 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:28.473619 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9mdfj" podStartSLOduration=2.035273829 podStartE2EDuration="3.473598033s" totalImagesPullingTime="1.438324204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:03:28.472818488 +0000 UTC m=+251.519345394" watchObservedRunningTime="2026-07-20 05:03:28.473598033 +0000 UTC m=+251.520124918" Jul 20 05:03:28.541428 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:28.541384 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:28.593875 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:28.593840 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:29.294691 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.294667 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:29.294773 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.294729 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:29.294855 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:29.294838 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle podName:1877c3b7-fa1e-49c5-a296-ef9478df42e8 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:33.29481781 +0000 UTC m=+256.341344698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle") pod "router-default-64bbb4d846-ftbct" (UID: "1877c3b7-fa1e-49c5-a296-ef9478df42e8") : configmap references non-existent config key: service-ca.crt Jul 20 05:03:29.294913 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:29.294878 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 05:03:29.294948 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:29.294931 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs podName:1877c3b7-fa1e-49c5-a296-ef9478df42e8 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:33.294919351 +0000 UTC m=+256.341446239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs") pod "router-default-64bbb4d846-ftbct" (UID: "1877c3b7-fa1e-49c5-a296-ef9478df42e8") : secret "router-metrics-certs-default" not found Jul 20 05:03:29.396144 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.396100 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-kndfv\" (UID: \"7f26fac3-f0a1-4461-8600-bf0a805d6cc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" Jul 20 05:03:29.396342 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.396233 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:29.396342 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:29.396251 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 05:03:29.396342 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:29.396319 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls podName:7f26fac3-f0a1-4461-8600-bf0a805d6cc6 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:33.39629989 +0000 UTC m=+256.442826780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-kndfv" (UID: "7f26fac3-f0a1-4461-8600-bf0a805d6cc6") : secret "samples-operator-tls" not found Jul 20 05:03:29.396342 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:29.396328 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 05:03:29.396561 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:29.396363 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls podName:4d64666c-dff4-424e-a774-9f90c94a206c nodeName:}" failed. No retries permitted until 2026-07-20 05:03:33.396350672 +0000 UTC m=+256.442877796 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-td56c" (UID: "4d64666c-dff4-424e-a774-9f90c94a206c") : secret "cluster-monitoring-operator-tls" not found Jul 20 05:03:29.464515 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.464482 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/0.log" Jul 20 05:03:29.464934 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.464532 2576 generic.go:356] "Generic (PLEG): container finished" podID="240fe5fe-623e-4495-a594-63f72c28f6bd" containerID="c897e5495e535919d67e46f6919409cc3d249a1771fb0581e69c04b542622674" exitCode=255 Jul 20 05:03:29.464934 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.464629 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" event={"ID":"240fe5fe-623e-4495-a594-63f72c28f6bd","Type":"ContainerDied","Data":"c897e5495e535919d67e46f6919409cc3d249a1771fb0581e69c04b542622674"} Jul 20 05:03:29.464934 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.464866 2576 scope.go:117] "RemoveContainer" containerID="c897e5495e535919d67e46f6919409cc3d249a1771fb0581e69c04b542622674" Jul 20 05:03:29.466227 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.466204 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-c9l2r" event={"ID":"9661ba55-2be6-499e-a1af-a5c0cb54f376","Type":"ContainerStarted","Data":"804a08922d4d12c961754251317f46b2adaaec152d0858ca49888613c757a9ce"} Jul 20 05:03:29.468061 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.468032 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" event={"ID":"dced5a3f-757b-47aa-b3b2-2ada2072603f","Type":"ContainerStarted","Data":"33b11697f418411c5c3d397ad108c1a2808c73f6c6147c1843c88eb09678de8a"} Jul 20 05:03:29.493618 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.493572 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-c9l2r" podStartSLOduration=1.511006664 podStartE2EDuration="4.493557978s" totalImagesPullingTime="2.982551314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:03:29.492524872 +0000 UTC m=+252.539051804" watchObservedRunningTime="2026-07-20 05:03:29.493557978 +0000 UTC m=+252.540084861" Jul 20 05:03:29.498352 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.498201 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 05:03:29.498487 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:29.498373 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 20 05:03:29.498487 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:29.498458 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs podName:06a59c1d-4c72-423f-9e8d-27db0a919dba nodeName:}" failed. No retries permitted until 2026-07-20 05:05:31.49843809 +0000 UTC m=+374.544964980 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs") pod "network-metrics-daemon-gnmjm" (UID: "06a59c1d-4c72-423f-9e8d-27db0a919dba") : secret "metrics-daemon-secret" not found Jul 20 05:03:29.507288 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.507240 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" podStartSLOduration=1.3012162 podStartE2EDuration="4.507222749s" totalImagesPullingTime="3.206006549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:03:29.506396644 +0000 UTC m=+252.552923551" watchObservedRunningTime="2026-07-20 05:03:29.507222749 +0000 UTC m=+252.553749656" Jul 20 05:03:29.682456 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:29.682421 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:30.382011 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.381981 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vddgs"] Jul 20 05:03:30.385233 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.385214 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.387437 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.387412 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 20 05:03:30.387548 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.387473 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 20 05:03:30.387717 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.387698 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-28fj9\"" Jul 20 05:03:30.393873 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.393849 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vddgs"] Jul 20 05:03:30.472976 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.472933 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:03:30.473419 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.473363 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/0.log" Jul 20 05:03:30.473478 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.473418 2576 generic.go:356] "Generic (PLEG): container finished" podID="240fe5fe-623e-4495-a594-63f72c28f6bd" containerID="4f923be05a7c81f07118b1fddeba9f9b4299b7f68a695fe02a6606a600da8798" exitCode=255 Jul 20 05:03:30.473531 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.473505 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" event={"ID":"240fe5fe-623e-4495-a594-63f72c28f6bd","Type":"ContainerDied","Data":"4f923be05a7c81f07118b1fddeba9f9b4299b7f68a695fe02a6606a600da8798"} Jul 20 05:03:30.473588 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.473565 2576 scope.go:117] "RemoveContainer" containerID="c897e5495e535919d67e46f6919409cc3d249a1771fb0581e69c04b542622674" Jul 20 05:03:30.473815 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.473793 2576 scope.go:117] "RemoveContainer" containerID="4f923be05a7c81f07118b1fddeba9f9b4299b7f68a695fe02a6606a600da8798" Jul 20 05:03:30.474256 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:30.474018 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-q8k7n_openshift-console-operator(240fe5fe-623e-4495-a594-63f72c28f6bd)\"" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" podUID="240fe5fe-623e-4495-a594-63f72c28f6bd" Jul 20 05:03:30.507795 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.507725 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj2qb\" (UniqueName: \"kubernetes.io/projected/a629203c-afcc-4207-8313-26f75cdc20ad-kube-api-access-rj2qb\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.508048 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.508026 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a629203c-afcc-4207-8313-26f75cdc20ad-crio-socket\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.508124 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.508061 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a629203c-afcc-4207-8313-26f75cdc20ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.508561 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.508511 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a629203c-afcc-4207-8313-26f75cdc20ad-data-volume\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.508688 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.508661 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.609259 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.609210 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.609467 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:30.609392 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 05:03:30.609521 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:30.609474 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls podName:a629203c-afcc-4207-8313-26f75cdc20ad nodeName:}" failed. No retries permitted until 2026-07-20 05:03:31.109452029 +0000 UTC m=+254.155978932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vddgs" (UID: "a629203c-afcc-4207-8313-26f75cdc20ad") : secret "insights-runtime-extractor-tls" not found Jul 20 05:03:30.609578 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.609518 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj2qb\" (UniqueName: \"kubernetes.io/projected/a629203c-afcc-4207-8313-26f75cdc20ad-kube-api-access-rj2qb\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.609649 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.609631 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a629203c-afcc-4207-8313-26f75cdc20ad-crio-socket\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.609704 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.609662 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a629203c-afcc-4207-8313-26f75cdc20ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.609704 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.609690 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a629203c-afcc-4207-8313-26f75cdc20ad-data-volume\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.609800 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:30.609725 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 05:03:30.609800 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:30.609787 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a629203c-afcc-4207-8313-26f75cdc20ad-kube-rbac-proxy-cm podName:a629203c-afcc-4207-8313-26f75cdc20ad nodeName:}" failed. No retries permitted until 2026-07-20 05:03:31.109773738 +0000 UTC m=+254.156300622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a629203c-afcc-4207-8313-26f75cdc20ad-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vddgs" (UID: "a629203c-afcc-4207-8313-26f75cdc20ad") : configmap "kube-rbac-proxy" not found Jul 20 05:03:30.609800 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.609723 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a629203c-afcc-4207-8313-26f75cdc20ad-crio-socket\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.610025 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.610009 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a629203c-afcc-4207-8313-26f75cdc20ad-data-volume\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.617236 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.617206 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj2qb\" (UniqueName: \"kubernetes.io/projected/a629203c-afcc-4207-8313-26f75cdc20ad-kube-api-access-rj2qb\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:30.931171 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:30.931139 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:31.030673 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:31.030633 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:31.114781 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:31.114745 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a629203c-afcc-4207-8313-26f75cdc20ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:31.114975 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:31.114805 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:31.115025 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:31.114976 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 05:03:31.115066 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:31.115055 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls podName:a629203c-afcc-4207-8313-26f75cdc20ad nodeName:}" failed. No retries permitted until 2026-07-20 05:03:32.115038204 +0000 UTC m=+255.161565089 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vddgs" (UID: "a629203c-afcc-4207-8313-26f75cdc20ad") : secret "insights-runtime-extractor-tls" not found Jul 20 05:03:31.115334 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:31.115315 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a629203c-afcc-4207-8313-26f75cdc20ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:31.392248 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:31.392217 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:31.468993 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:31.468947 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:31.477781 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:31.477761 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:03:31.478156 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:31.478100 2576 scope.go:117] "RemoveContainer" containerID="4f923be05a7c81f07118b1fddeba9f9b4299b7f68a695fe02a6606a600da8798" Jul 20 05:03:31.478273 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:31.478255 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-q8k7n_openshift-console-operator(240fe5fe-623e-4495-a594-63f72c28f6bd)\"" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" podUID="240fe5fe-623e-4495-a594-63f72c28f6bd" Jul 20 05:03:31.669706 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:31.669631 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:31.772213 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:31.772182 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:32.123918 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:32.123867 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:32.124101 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:32.124034 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 05:03:32.124101 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:32.124089 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls podName:a629203c-afcc-4207-8313-26f75cdc20ad nodeName:}" failed. No retries permitted until 2026-07-20 05:03:34.124075202 +0000 UTC m=+257.170602088 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vddgs" (UID: "a629203c-afcc-4207-8313-26f75cdc20ad") : secret "insights-runtime-extractor-tls" not found Jul 20 05:03:32.517507 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:32.517478 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:32.566803 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:32.566770 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:33.165223 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.165197 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-js7gh_1edae688-9ce1-49e7-9a37-5d782b643b36/dns-node-resolver/0.log" Jul 20 05:03:33.268156 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.268119 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-swnz5"] Jul 20 05:03:33.272629 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.272610 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-swnz5" Jul 20 05:03:33.278355 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.278323 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-dwpmc\"" Jul 20 05:03:33.278500 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.278373 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 20 05:03:33.279495 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.279470 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 20 05:03:33.279761 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.279741 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 20 05:03:33.279870 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.279843 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 20 05:03:33.280163 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.280144 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-swnz5"] Jul 20 05:03:33.302225 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.302175 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:33.334052 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.334018 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9da2ffb4-2c77-444e-a0c5-0aa13420c7ad-signing-cabundle\") pod \"service-ca-58598695f7-swnz5\" (UID: \"9da2ffb4-2c77-444e-a0c5-0aa13420c7ad\") " pod="openshift-service-ca/service-ca-58598695f7-swnz5" Jul 20 05:03:33.334052 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.334050 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9da2ffb4-2c77-444e-a0c5-0aa13420c7ad-signing-key\") pod \"service-ca-58598695f7-swnz5\" (UID: \"9da2ffb4-2c77-444e-a0c5-0aa13420c7ad\") " pod="openshift-service-ca/service-ca-58598695f7-swnz5" Jul 20 05:03:33.334243 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.334085 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:33.334243 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.334146 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8k8c\" (UniqueName: \"kubernetes.io/projected/9da2ffb4-2c77-444e-a0c5-0aa13420c7ad-kube-api-access-r8k8c\") pod \"service-ca-58598695f7-swnz5\" (UID: \"9da2ffb4-2c77-444e-a0c5-0aa13420c7ad\") " pod="openshift-service-ca/service-ca-58598695f7-swnz5" Jul 20 05:03:33.334243 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:33.334211 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle podName:1877c3b7-fa1e-49c5-a296-ef9478df42e8 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:41.334194084 +0000 UTC m=+264.380720972 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle") pod "router-default-64bbb4d846-ftbct" (UID: "1877c3b7-fa1e-49c5-a296-ef9478df42e8") : configmap references non-existent config key: service-ca.crt Jul 20 05:03:33.334340 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.334243 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:33.334390 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:33.334375 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 05:03:33.334430 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:33.334420 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs podName:1877c3b7-fa1e-49c5-a296-ef9478df42e8 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:41.334407966 +0000 UTC m=+264.380934849 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs") pod "router-default-64bbb4d846-ftbct" (UID: "1877c3b7-fa1e-49c5-a296-ef9478df42e8") : secret "router-metrics-certs-default" not found Jul 20 05:03:33.350841 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.350812 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:33.435431 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.435344 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:33.435431 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.435415 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-kndfv\" (UID: \"7f26fac3-f0a1-4461-8600-bf0a805d6cc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" Jul 20 05:03:33.435703 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.435452 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9da2ffb4-2c77-444e-a0c5-0aa13420c7ad-signing-cabundle\") pod \"service-ca-58598695f7-swnz5\" (UID: \"9da2ffb4-2c77-444e-a0c5-0aa13420c7ad\") " pod="openshift-service-ca/service-ca-58598695f7-swnz5" Jul 20 05:03:33.435703 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.435475 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9da2ffb4-2c77-444e-a0c5-0aa13420c7ad-signing-key\") pod \"service-ca-58598695f7-swnz5\" (UID: \"9da2ffb4-2c77-444e-a0c5-0aa13420c7ad\") " pod="openshift-service-ca/service-ca-58598695f7-swnz5" Jul 20 05:03:33.435703 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:33.435516 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 05:03:33.435703 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.435524 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8k8c\" (UniqueName: \"kubernetes.io/projected/9da2ffb4-2c77-444e-a0c5-0aa13420c7ad-kube-api-access-r8k8c\") pod \"service-ca-58598695f7-swnz5\" (UID: \"9da2ffb4-2c77-444e-a0c5-0aa13420c7ad\") " pod="openshift-service-ca/service-ca-58598695f7-swnz5" Jul 20 05:03:33.435703 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:33.435583 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls podName:7f26fac3-f0a1-4461-8600-bf0a805d6cc6 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:41.4355624 +0000 UTC m=+264.482089284 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-kndfv" (UID: "7f26fac3-f0a1-4461-8600-bf0a805d6cc6") : secret "samples-operator-tls" not found Jul 20 05:03:33.435703 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:33.435517 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 05:03:33.435703 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:33.435683 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls podName:4d64666c-dff4-424e-a774-9f90c94a206c nodeName:}" failed. No retries permitted until 2026-07-20 05:03:41.435659125 +0000 UTC m=+264.482186021 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-td56c" (UID: "4d64666c-dff4-424e-a774-9f90c94a206c") : secret "cluster-monitoring-operator-tls" not found Jul 20 05:03:33.436160 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.436140 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9da2ffb4-2c77-444e-a0c5-0aa13420c7ad-signing-cabundle\") pod \"service-ca-58598695f7-swnz5\" (UID: \"9da2ffb4-2c77-444e-a0c5-0aa13420c7ad\") " pod="openshift-service-ca/service-ca-58598695f7-swnz5" Jul 20 05:03:33.437942 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.437925 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9da2ffb4-2c77-444e-a0c5-0aa13420c7ad-signing-key\") pod \"service-ca-58598695f7-swnz5\" (UID: \"9da2ffb4-2c77-444e-a0c5-0aa13420c7ad\") " pod="openshift-service-ca/service-ca-58598695f7-swnz5" Jul 20 05:03:33.443554 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.443528 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8k8c\" (UniqueName: \"kubernetes.io/projected/9da2ffb4-2c77-444e-a0c5-0aa13420c7ad-kube-api-access-r8k8c\") pod \"service-ca-58598695f7-swnz5\" (UID: \"9da2ffb4-2c77-444e-a0c5-0aa13420c7ad\") " pod="openshift-service-ca/service-ca-58598695f7-swnz5" Jul 20 05:03:33.582355 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.582314 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-swnz5" Jul 20 05:03:33.707272 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:33.707194 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-swnz5"] Jul 20 05:03:33.710011 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:03:33.709983 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9da2ffb4_2c77_444e_a0c5_0aa13420c7ad.slice/crio-da641f0478658ad6df681a08c20b662bbdf45a63a5b04e3f4587ba70c21be79c WatchSource:0}: Error finding container da641f0478658ad6df681a08c20b662bbdf45a63a5b04e3f4587ba70c21be79c: Status 404 returned error can't find the container with id da641f0478658ad6df681a08c20b662bbdf45a63a5b04e3f4587ba70c21be79c Jul 20 05:03:34.141433 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:34.141400 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:34.141610 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:34.141562 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 05:03:34.141647 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:34.141628 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls podName:a629203c-afcc-4207-8313-26f75cdc20ad nodeName:}" failed. No retries permitted until 2026-07-20 05:03:38.141612623 +0000 UTC m=+261.188139507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vddgs" (UID: "a629203c-afcc-4207-8313-26f75cdc20ad") : secret "insights-runtime-extractor-tls" not found Jul 20 05:03:34.165966 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:34.165940 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p29tq_42007563-e764-4689-a28f-872b48cd1e8b/node-ca/0.log" Jul 20 05:03:34.199430 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:34.199397 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:34.488815 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:34.488782 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-swnz5" event={"ID":"9da2ffb4-2c77-444e-a0c5-0aa13420c7ad","Type":"ContainerStarted","Data":"a7c8145fea0838032df58a5bb19a2c2266359cd9e092fc012398e9610842d42c"} Jul 20 05:03:34.489020 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:34.488836 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-swnz5" event={"ID":"9da2ffb4-2c77-444e-a0c5-0aa13420c7ad","Type":"ContainerStarted","Data":"da641f0478658ad6df681a08c20b662bbdf45a63a5b04e3f4587ba70c21be79c"} Jul 20 05:03:34.506531 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:34.506472 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-swnz5" podStartSLOduration=1.50645048 podStartE2EDuration="1.50645048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:03:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:03:34.504805397 +0000 UTC m=+257.551332324" watchObservedRunningTime="2026-07-20 05:03:34.50645048 +0000 UTC m=+257.552977388" Jul 20 05:03:34.965685 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:34.965597 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sf226_03c3ada9-2806-4b6e-9e90-ee27aed55433/serve-healthcheck-canary/0.log" Jul 20 05:03:35.434848 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:35.434784 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:35.617125 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:35.617091 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:35.718491 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:35.718407 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:35.866464 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:35.866431 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:35.940940 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:35.940905 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:35.941128 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:35.940984 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:35.941340 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:35.941326 2576 scope.go:117] "RemoveContainer" containerID="4f923be05a7c81f07118b1fddeba9f9b4299b7f68a695fe02a6606a600da8798" Jul 20 05:03:35.941522 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:35.941502 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-q8k7n_openshift-console-operator(240fe5fe-623e-4495-a594-63f72c28f6bd)\"" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" podUID="240fe5fe-623e-4495-a594-63f72c28f6bd" Jul 20 05:03:35.970381 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:35.970288 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:36.337427 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:36.337394 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:36.437728 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:36.437694 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:36.487833 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:36.487793 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:36.735288 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:36.735256 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:36.937327 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:36.937295 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:37.322862 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:37.322831 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:37.938572 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:37.938532 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:38.173848 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:38.173814 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:38.174074 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:38.174020 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 05:03:38.174149 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:38.174134 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls podName:a629203c-afcc-4207-8313-26f75cdc20ad nodeName:}" failed. No retries permitted until 2026-07-20 05:03:46.174111413 +0000 UTC m=+269.220638300 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vddgs" (UID: "a629203c-afcc-4207-8313-26f75cdc20ad") : secret "insights-runtime-extractor-tls" not found Jul 20 05:03:38.337248 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:38.337215 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:38.388090 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:38.388050 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:38.539630 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:38.539555 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:38.737859 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:38.737829 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:38.937577 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:38.937547 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:39.338780 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:39.338748 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:39.596033 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:39.595930 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:39.734132 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:39.734074 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:40.345012 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:40.344978 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:40.445586 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:40.445549 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:40.547254 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:40.547221 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:40.605779 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:40.605700 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:40.944211 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:40.944109 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:41.139053 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:41.139015 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:41.338968 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:41.338926 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:41.400492 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:41.400457 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:41.401004 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:41.400521 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:41.401004 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:41.400679 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle podName:1877c3b7-fa1e-49c5-a296-ef9478df42e8 nodeName:}" failed. No retries permitted until 2026-07-20 05:03:57.400652525 +0000 UTC m=+280.447179414 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle") pod "router-default-64bbb4d846-ftbct" (UID: "1877c3b7-fa1e-49c5-a296-ef9478df42e8") : configmap references non-existent config key: service-ca.crt Jul 20 05:03:41.402843 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:41.402818 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1877c3b7-fa1e-49c5-a296-ef9478df42e8-metrics-certs\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:41.501497 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:41.501460 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:41.501677 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:41.501531 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-kndfv\" (UID: \"7f26fac3-f0a1-4461-8600-bf0a805d6cc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" Jul 20 05:03:41.501677 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:41.501624 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 05:03:41.501766 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:41.501702 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls podName:4d64666c-dff4-424e-a774-9f90c94a206c nodeName:}" failed. No retries permitted until 2026-07-20 05:03:57.501682339 +0000 UTC m=+280.548209224 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-td56c" (UID: "4d64666c-dff4-424e-a774-9f90c94a206c") : secret "cluster-monitoring-operator-tls" not found Jul 20 05:03:41.503949 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:41.503925 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f26fac3-f0a1-4461-8600-bf0a805d6cc6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-kndfv\" (UID: \"7f26fac3-f0a1-4461-8600-bf0a805d6cc6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" Jul 20 05:03:41.621602 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:41.621514 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" Jul 20 05:03:41.746434 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:41.746402 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv"] Jul 20 05:03:42.335658 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:42.335617 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:42.521082 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:42.521048 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" event={"ID":"7f26fac3-f0a1-4461-8600-bf0a805d6cc6","Type":"ContainerStarted","Data":"9d5f0f56d715620dbfad22222e295695e5030a04dca7a1399e704520c1d8dac7"} Jul 20 05:03:42.737194 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:42.737160 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:42.960509 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:42.960450 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:43.010275 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:43.010196 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:43.139095 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:43.139049 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:43.532565 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:43.532544 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:44.134589 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:44.134562 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:44.529897 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:44.529854 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" event={"ID":"7f26fac3-f0a1-4461-8600-bf0a805d6cc6","Type":"ContainerStarted","Data":"a17af1dcb81e3daca8821dbe1add4c2e81edcac47c587fb1d825217f950f7834"} Jul 20 05:03:44.530133 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:44.529917 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" event={"ID":"7f26fac3-f0a1-4461-8600-bf0a805d6cc6","Type":"ContainerStarted","Data":"ea08e40a92597ea097f17f9553afbfc964d6c90eb7df3b03b84729497d1a36ce"} Jul 20 05:03:44.545675 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:44.545626 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-kndfv" podStartSLOduration=17.853766341 podStartE2EDuration="19.545608104s" totalImagesPullingTime="1.691841763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:03:44.544542259 +0000 UTC m=+267.591069184" watchObservedRunningTime="2026-07-20 05:03:44.545608104 +0000 UTC m=+267.592135006" Jul 20 05:03:44.743696 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:44.743659 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:44.936985 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:44.936889 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:45.334620 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:45.334587 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:45.534260 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:45.534216 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:45.935714 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:45.935685 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:46.134463 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:46.134429 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:46.245126 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:46.245082 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:46.247370 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:46.247345 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a629203c-afcc-4207-8313-26f75cdc20ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vddgs\" (UID: \"a629203c-afcc-4207-8313-26f75cdc20ad\") " pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:46.295521 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:46.295490 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vddgs" Jul 20 05:03:46.421100 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:46.421065 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vddgs"] Jul 20 05:03:46.424454 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:03:46.424428 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda629203c_afcc_4207_8313_26f75cdc20ad.slice/crio-dd288ac131cf7d296991087ead6144584594d563ed944bebc19830a86c9b40fd WatchSource:0}: Error finding container dd288ac131cf7d296991087ead6144584594d563ed944bebc19830a86c9b40fd: Status 404 returned error can't find the container with id dd288ac131cf7d296991087ead6144584594d563ed944bebc19830a86c9b40fd Jul 20 05:03:46.538077 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:46.537981 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vddgs" event={"ID":"a629203c-afcc-4207-8313-26f75cdc20ad","Type":"ContainerStarted","Data":"dd288ac131cf7d296991087ead6144584594d563ed944bebc19830a86c9b40fd"} Jul 20 05:03:50.647590 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:50.647558 2576 scope.go:117] "RemoveContainer" containerID="4f923be05a7c81f07118b1fddeba9f9b4299b7f68a695fe02a6606a600da8798" Jul 20 05:03:51.396013 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:51.395983 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:51.494689 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:51.494652 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:51.557205 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:51.557178 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:03:51.557377 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:51.557262 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" event={"ID":"240fe5fe-623e-4495-a594-63f72c28f6bd","Type":"ContainerStarted","Data":"bf7d0ab85231c4e2597c73555eeee33ec43345eb5751a0e9ccae2b661a6fef69"} Jul 20 05:03:51.557592 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:51.557571 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:51.558867 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:51.558835 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vddgs" event={"ID":"a629203c-afcc-4207-8313-26f75cdc20ad","Type":"ContainerStarted","Data":"51cc8cd94e1c3eb1d8dd2ae73e112850d6568af9ed6c3a238dea054e5d19a973"} Jul 20 05:03:51.573661 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:51.573617 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" podStartSLOduration=23.589466326 podStartE2EDuration="26.573603953s" totalImagesPullingTime="2.984137627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:03:51.571991519 +0000 UTC m=+274.618518444" watchObservedRunningTime="2026-07-20 05:03:51.573603953 +0000 UTC m=+274.620130860" Jul 20 05:03:52.004323 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:52.004266 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-q8k7n" Jul 20 05:03:52.171821 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:52.171785 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:52.272584 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:52.272509 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:52.323101 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:52.323044 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:52.471877 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:52.471835 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:52.647278 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:52.647196 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:52.698637 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:52.698604 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:53.075238 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:53.075186 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:53.129090 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:53.129062 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:53.257629 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:53.257594 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:53.477380 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:53.477311 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:53.529117 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:53.529089 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:53.858429 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:53.858392 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:53.979240 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:53.979208 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:54.087906 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.087877 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:54.708717 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.708678 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-z65tl"] Jul 20 05:03:54.721204 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.721166 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-559bbfdc6f-vjzzs"] Jul 20 05:03:54.721431 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.721287 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-z65tl" Jul 20 05:03:54.721526 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:03:54.721434 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" podUID="12256a20-8d87-4866-afcd-51b5ecfb3936" Jul 20 05:03:54.722318 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.722293 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-z65tl"] Jul 20 05:03:54.723890 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.723869 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 20 05:03:54.724196 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.724171 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 20 05:03:54.724716 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.724531 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-n4zlr\"" Jul 20 05:03:54.793217 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.793130 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:54.810208 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.810183 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7899876bcf-zm2cl"] Jul 20 05:03:54.813704 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.813681 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:54.822385 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.822359 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb6g6\" (UniqueName: \"kubernetes.io/projected/91155ba9-0eab-446c-a27e-a2c2d7321584-kube-api-access-tb6g6\") pod \"downloads-5bc67dcd46-z65tl\" (UID: \"91155ba9-0eab-446c-a27e-a2c2d7321584\") " pod="openshift-console/downloads-5bc67dcd46-z65tl" Jul 20 05:03:54.825064 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.825040 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7899876bcf-zm2cl"] Jul 20 05:03:54.886665 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.886640 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:54.922843 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.922825 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c73e5531-c6cc-4512-bdef-f859d992f7ec-registry-certificates\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:54.922980 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.922863 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c73e5531-c6cc-4512-bdef-f859d992f7ec-installation-pull-secrets\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:54.922980 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.922888 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c73e5531-c6cc-4512-bdef-f859d992f7ec-bound-sa-token\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:54.923090 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.923008 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vq4h\" (UniqueName: \"kubernetes.io/projected/c73e5531-c6cc-4512-bdef-f859d992f7ec-kube-api-access-7vq4h\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:54.923090 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.923046 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c73e5531-c6cc-4512-bdef-f859d992f7ec-ca-trust-extracted\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:54.923090 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.923083 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb6g6\" (UniqueName: \"kubernetes.io/projected/91155ba9-0eab-446c-a27e-a2c2d7321584-kube-api-access-tb6g6\") pod \"downloads-5bc67dcd46-z65tl\" (UID: \"91155ba9-0eab-446c-a27e-a2c2d7321584\") " pod="openshift-console/downloads-5bc67dcd46-z65tl" Jul 20 05:03:54.923175 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.923161 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c73e5531-c6cc-4512-bdef-f859d992f7ec-image-registry-private-configuration\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:54.923205 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.923184 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c73e5531-c6cc-4512-bdef-f859d992f7ec-trusted-ca\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:54.923290 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.923271 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c73e5531-c6cc-4512-bdef-f859d992f7ec-registry-tls\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:54.932242 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:54.932214 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb6g6\" (UniqueName: \"kubernetes.io/projected/91155ba9-0eab-446c-a27e-a2c2d7321584-kube-api-access-tb6g6\") pod \"downloads-5bc67dcd46-z65tl\" (UID: \"91155ba9-0eab-446c-a27e-a2c2d7321584\") " pod="openshift-console/downloads-5bc67dcd46-z65tl" Jul 20 05:03:55.013440 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.013410 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:55.024152 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.024125 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c73e5531-c6cc-4512-bdef-f859d992f7ec-image-registry-private-configuration\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.024251 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.024157 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c73e5531-c6cc-4512-bdef-f859d992f7ec-trusted-ca\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.024412 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.024388 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c73e5531-c6cc-4512-bdef-f859d992f7ec-registry-tls\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.024469 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.024451 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c73e5531-c6cc-4512-bdef-f859d992f7ec-registry-certificates\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.024531 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.024508 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c73e5531-c6cc-4512-bdef-f859d992f7ec-installation-pull-secrets\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.024568 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.024549 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c73e5531-c6cc-4512-bdef-f859d992f7ec-bound-sa-token\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.024627 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.024612 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vq4h\" (UniqueName: \"kubernetes.io/projected/c73e5531-c6cc-4512-bdef-f859d992f7ec-kube-api-access-7vq4h\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.024693 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.024655 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c73e5531-c6cc-4512-bdef-f859d992f7ec-ca-trust-extracted\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.025543 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.025512 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c73e5531-c6cc-4512-bdef-f859d992f7ec-ca-trust-extracted\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.025733 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.025710 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c73e5531-c6cc-4512-bdef-f859d992f7ec-registry-certificates\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.026241 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.026215 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c73e5531-c6cc-4512-bdef-f859d992f7ec-trusted-ca\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.027553 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.027225 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c73e5531-c6cc-4512-bdef-f859d992f7ec-image-registry-private-configuration\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.027553 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.027486 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c73e5531-c6cc-4512-bdef-f859d992f7ec-installation-pull-secrets\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.031294 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.031273 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c73e5531-c6cc-4512-bdef-f859d992f7ec-registry-tls\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.032849 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.032830 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-z65tl" Jul 20 05:03:55.038164 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.038141 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vq4h\" (UniqueName: \"kubernetes.io/projected/c73e5531-c6cc-4512-bdef-f859d992f7ec-kube-api-access-7vq4h\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.038164 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.038147 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c73e5531-c6cc-4512-bdef-f859d992f7ec-bound-sa-token\") pod \"image-registry-7899876bcf-zm2cl\" (UID: \"c73e5531-c6cc-4512-bdef-f859d992f7ec\") " pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.112597 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.112511 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:55.137697 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.137466 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vkwrd\"" Jul 20 05:03:55.146205 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.146162 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.184347 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.184228 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-z65tl"] Jul 20 05:03:55.189170 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:03:55.189143 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91155ba9_0eab_446c_a27e_a2c2d7321584.slice/crio-74c9f0e81fb2e5f5e22e69461a49b0a537611e3f7380ea4df1e37979352a8410 WatchSource:0}: Error finding container 74c9f0e81fb2e5f5e22e69461a49b0a537611e3f7380ea4df1e37979352a8410: Status 404 returned error can't find the container with id 74c9f0e81fb2e5f5e22e69461a49b0a537611e3f7380ea4df1e37979352a8410 Jul 20 05:03:55.276998 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.276863 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7899876bcf-zm2cl"] Jul 20 05:03:55.279368 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:03:55.279344 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73e5531_c6cc_4512_bdef_f859d992f7ec.slice/crio-0a30ea13e4cef945ec5058e78553454fca4bc50094189266d1da1602f2e7be20 WatchSource:0}: Error finding container 0a30ea13e4cef945ec5058e78553454fca4bc50094189266d1da1602f2e7be20: Status 404 returned error can't find the container with id 0a30ea13e4cef945ec5058e78553454fca4bc50094189266d1da1602f2e7be20 Jul 20 05:03:55.575420 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.575335 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vddgs" event={"ID":"a629203c-afcc-4207-8313-26f75cdc20ad","Type":"ContainerStarted","Data":"fc0c4f1d1af2c2274ff043e061cea00f5d756c8e7f4eaeee7c5cc975862a003b"} Jul 20 05:03:55.577028 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.577000 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" event={"ID":"c73e5531-c6cc-4512-bdef-f859d992f7ec","Type":"ContainerStarted","Data":"6e068d84fc4507ff51041b00ab4637892b8bac3737671aa6bbb08a9b6105283a"} Jul 20 05:03:55.577147 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.577049 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" event={"ID":"c73e5531-c6cc-4512-bdef-f859d992f7ec","Type":"ContainerStarted","Data":"0a30ea13e4cef945ec5058e78553454fca4bc50094189266d1da1602f2e7be20"} Jul 20 05:03:55.577207 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.577182 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:03:55.578422 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.578392 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-z65tl" event={"ID":"91155ba9-0eab-446c-a27e-a2c2d7321584","Type":"ContainerStarted","Data":"74c9f0e81fb2e5f5e22e69461a49b0a537611e3f7380ea4df1e37979352a8410"} Jul 20 05:03:55.578528 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.578430 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 05:03:55.586016 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.585995 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 05:03:55.597718 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.597659 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" podStartSLOduration=1.597638817 podStartE2EDuration="1.597638817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:03:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:03:55.595878264 +0000 UTC m=+278.642405194" watchObservedRunningTime="2026-07-20 05:03:55.597638817 +0000 UTC m=+278.644165723" Jul 20 05:03:55.630154 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.630104 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:55.694147 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.694111 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:03:55.732096 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.732062 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-bound-sa-token\") pod \"12256a20-8d87-4866-afcd-51b5ecfb3936\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " Jul 20 05:03:55.732096 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.732107 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-certificates\") pod \"12256a20-8d87-4866-afcd-51b5ecfb3936\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " Jul 20 05:03:55.732343 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.732152 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12256a20-8d87-4866-afcd-51b5ecfb3936-installation-pull-secrets\") pod \"12256a20-8d87-4866-afcd-51b5ecfb3936\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " Jul 20 05:03:55.732343 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.732238 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzs2s\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-kube-api-access-bzs2s\") pod \"12256a20-8d87-4866-afcd-51b5ecfb3936\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " Jul 20 05:03:55.732343 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.732288 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12256a20-8d87-4866-afcd-51b5ecfb3936-ca-trust-extracted\") pod \"12256a20-8d87-4866-afcd-51b5ecfb3936\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " Jul 20 05:03:55.732343 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.732324 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12256a20-8d87-4866-afcd-51b5ecfb3936-trusted-ca\") pod \"12256a20-8d87-4866-afcd-51b5ecfb3936\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " Jul 20 05:03:55.732533 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.732402 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12256a20-8d87-4866-afcd-51b5ecfb3936-image-registry-private-configuration\") pod \"12256a20-8d87-4866-afcd-51b5ecfb3936\" (UID: \"12256a20-8d87-4866-afcd-51b5ecfb3936\") " Jul 20 05:03:55.732604 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.732524 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "12256a20-8d87-4866-afcd-51b5ecfb3936" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:03:55.733552 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.732919 2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-certificates\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:03:55.733552 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.733510 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12256a20-8d87-4866-afcd-51b5ecfb3936-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "12256a20-8d87-4866-afcd-51b5ecfb3936" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:03:55.733761 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.733726 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12256a20-8d87-4866-afcd-51b5ecfb3936-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "12256a20-8d87-4866-afcd-51b5ecfb3936" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:03:55.735347 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.735303 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12256a20-8d87-4866-afcd-51b5ecfb3936-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "12256a20-8d87-4866-afcd-51b5ecfb3936" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 05:03:55.735347 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.735337 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-kube-api-access-bzs2s" (OuterVolumeSpecName: "kube-api-access-bzs2s") pod "12256a20-8d87-4866-afcd-51b5ecfb3936" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936"). InnerVolumeSpecName "kube-api-access-bzs2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:03:55.735525 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.735463 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12256a20-8d87-4866-afcd-51b5ecfb3936-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "12256a20-8d87-4866-afcd-51b5ecfb3936" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 05:03:55.735685 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.735666 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "12256a20-8d87-4866-afcd-51b5ecfb3936" (UID: "12256a20-8d87-4866-afcd-51b5ecfb3936"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:03:55.834525 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.834317 2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12256a20-8d87-4866-afcd-51b5ecfb3936-ca-trust-extracted\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:03:55.834525 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.834392 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12256a20-8d87-4866-afcd-51b5ecfb3936-trusted-ca\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:03:55.834525 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.834409 2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12256a20-8d87-4866-afcd-51b5ecfb3936-image-registry-private-configuration\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:03:55.834525 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.834425 2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-bound-sa-token\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:03:55.834525 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.834438 2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12256a20-8d87-4866-afcd-51b5ecfb3936-installation-pull-secrets\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:03:55.834525 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:55.834451 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzs2s\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-kube-api-access-bzs2s\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:03:56.583248 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:56.583206 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559bbfdc6f-vjzzs" Jul 20 05:03:56.616718 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:56.616687 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-559bbfdc6f-vjzzs"] Jul 20 05:03:56.619763 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:56.619708 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-559bbfdc6f-vjzzs"] Jul 20 05:03:56.744230 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:56.744200 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12256a20-8d87-4866-afcd-51b5ecfb3936-registry-tls\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:03:57.449503 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:57.449461 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:57.450382 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:57.450351 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1877c3b7-fa1e-49c5-a296-ef9478df42e8-service-ca-bundle\") pod \"router-default-64bbb4d846-ftbct\" (UID: \"1877c3b7-fa1e-49c5-a296-ef9478df42e8\") " pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:57.452443 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:57.452413 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:03:57.549917 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:57.549869 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:57.552718 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:57.552692 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d64666c-dff4-424e-a774-9f90c94a206c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-td56c\" (UID: \"4d64666c-dff4-424e-a774-9f90c94a206c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:57.652564 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:57.652529 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12256a20-8d87-4866-afcd-51b5ecfb3936" path="/var/lib/kubelet/pods/12256a20-8d87-4866-afcd-51b5ecfb3936/volumes" Jul 20 05:03:57.828387 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:57.828327 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" Jul 20 05:03:58.777300 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:58.777271 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64bbb4d846-ftbct"] Jul 20 05:03:58.780782 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:03:58.780754 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1877c3b7_fa1e_49c5_a296_ef9478df42e8.slice/crio-f2143e87024c2933daf333b6fac430bef3561a001ea9ef43958dbc16f6e3c15d WatchSource:0}: Error finding container f2143e87024c2933daf333b6fac430bef3561a001ea9ef43958dbc16f6e3c15d: Status 404 returned error can't find the container with id f2143e87024c2933daf333b6fac430bef3561a001ea9ef43958dbc16f6e3c15d Jul 20 05:03:58.792791 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:58.792768 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c"] Jul 20 05:03:58.794878 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:03:58.794845 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d64666c_dff4_424e_a774_9f90c94a206c.slice/crio-4d53b8fbf30dcd2ed7f6b5d17460b640ebd18dab71eac025fffb55596ed51eee WatchSource:0}: Error finding container 4d53b8fbf30dcd2ed7f6b5d17460b640ebd18dab71eac025fffb55596ed51eee: Status 404 returned error can't find the container with id 4d53b8fbf30dcd2ed7f6b5d17460b640ebd18dab71eac025fffb55596ed51eee Jul 20 05:03:59.597808 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:59.597751 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vddgs" event={"ID":"a629203c-afcc-4207-8313-26f75cdc20ad","Type":"ContainerStarted","Data":"46b953682aaf0537e20167e3c2af42110595844dc36cddf8aa90481da884e3c2"} Jul 20 05:03:59.599443 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:59.599408 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64bbb4d846-ftbct" event={"ID":"1877c3b7-fa1e-49c5-a296-ef9478df42e8","Type":"ContainerStarted","Data":"1532e7f44cf15d9268d4a2636077aadc3da0fabcf15fa4a6010b9c55962816a4"} Jul 20 05:03:59.599575 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:59.599455 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64bbb4d846-ftbct" event={"ID":"1877c3b7-fa1e-49c5-a296-ef9478df42e8","Type":"ContainerStarted","Data":"f2143e87024c2933daf333b6fac430bef3561a001ea9ef43958dbc16f6e3c15d"} Jul 20 05:03:59.600684 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:59.600639 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" event={"ID":"4d64666c-dff4-424e-a774-9f90c94a206c","Type":"ContainerStarted","Data":"4d53b8fbf30dcd2ed7f6b5d17460b640ebd18dab71eac025fffb55596ed51eee"} Jul 20 05:03:59.614900 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:59.614847 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vddgs" podStartSLOduration=17.487054271 podStartE2EDuration="29.614831609s" totalImagesPullingTime="12.127777338s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 05:03:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:03:59.613327921 +0000 UTC m=+282.659854846" watchObservedRunningTime="2026-07-20 05:03:59.614831609 +0000 UTC m=+282.661358493" Jul 20 05:03:59.631692 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:03:59.631638 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-64bbb4d846-ftbct" podStartSLOduration=34.631618831 podStartE2EDuration="34.631618831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:03:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:03:59.630498909 +0000 UTC m=+282.677025816" watchObservedRunningTime="2026-07-20 05:03:59.631618831 +0000 UTC m=+282.678145739" Jul 20 05:04:00.094193 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:00.094102 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:00.155811 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:00.155757 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:00.453672 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:00.453544 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:04:00.456626 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:00.456587 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:04:00.463760 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:00.463724 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:00.605273 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:00.605199 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:04:00.606501 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:00.606473 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-64bbb4d846-ftbct" Jul 20 05:04:01.220302 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:01.220272 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:01.320805 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:01.320759 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:01.401876 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:01.401840 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:01.466666 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:01.466630 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:01.611697 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:01.611662 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" event={"ID":"4d64666c-dff4-424e-a774-9f90c94a206c","Type":"ContainerStarted","Data":"5212b5244cf5eae8c12edb0888d607894646bb44b1a477da634accade33249ba"} Jul 20 05:04:01.628657 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:01.628587 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-td56c" podStartSLOduration=34.558387934 podStartE2EDuration="36.628564248s" totalImagesPullingTime="2.070176314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:04:01.626190605 +0000 UTC m=+284.672717514" watchObservedRunningTime="2026-07-20 05:04:01.628564248 +0000 UTC m=+284.675091152" Jul 20 05:04:01.888696 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:01.888608 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:01.951548 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:01.951505 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:02.901658 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:02.901615 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 05:04:02.904208 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:02.904171 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a13f44c-570d-4fa6-96c6-bcdead54f7ec-metrics-tls\") pod \"dns-default-cvb6r\" (UID: \"6a13f44c-570d-4fa6-96c6-bcdead54f7ec\") " pod="openshift-dns/dns-default-cvb6r" Jul 20 05:04:02.950816 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:02.950782 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jlw9p\"" Jul 20 05:04:02.958906 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:02.958872 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cvb6r" Jul 20 05:04:03.157624 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:03.157593 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cvb6r"] Jul 20 05:04:03.160717 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:04:03.160682 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a13f44c_570d_4fa6_96c6_bcdead54f7ec.slice/crio-312be9bfe4b6b37f0d68cb19c98714b0ed3bae5c13ad93a399f02dcfc98f8146 WatchSource:0}: Error finding container 312be9bfe4b6b37f0d68cb19c98714b0ed3bae5c13ad93a399f02dcfc98f8146: Status 404 returned error can't find the container with id 312be9bfe4b6b37f0d68cb19c98714b0ed3bae5c13ad93a399f02dcfc98f8146 Jul 20 05:04:03.622822 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:03.622783 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cvb6r" event={"ID":"6a13f44c-570d-4fa6-96c6-bcdead54f7ec","Type":"ContainerStarted","Data":"312be9bfe4b6b37f0d68cb19c98714b0ed3bae5c13ad93a399f02dcfc98f8146"} Jul 20 05:04:06.409865 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:06.409825 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:06.509945 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:06.509912 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:06.699191 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:06.699112 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:08.759406 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:08.759369 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:09.175571 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:09.175488 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:11.555601 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.555561 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:11.722792 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.722744 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:11.798398 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.798344 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:11.804357 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.803626 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh"] Jul 20 05:04:11.831122 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.830673 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-qvbb9"] Jul 20 05:04:11.832194 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.832170 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:11.836304 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.835692 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-b7srh\"" Jul 20 05:04:11.836304 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.835986 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 20 05:04:11.836304 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.836283 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 20 05:04:11.836531 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.836476 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 20 05:04:11.849658 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.848786 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh"] Jul 20 05:04:11.849658 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.848837 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mh57v"] Jul 20 05:04:11.850675 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.850039 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:11.852931 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.852688 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 20 05:04:11.852931 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.852901 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 20 05:04:11.853377 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.853354 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 20 05:04:11.853553 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.853539 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hp2rg\"" Jul 20 05:04:11.865643 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.865261 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-qvbb9"] Jul 20 05:04:11.865643 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.865401 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:11.867981 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.867946 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 20 05:04:11.868342 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.868328 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wkpjg\"" Jul 20 05:04:11.868517 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.868496 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 20 05:04:11.868705 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.868626 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 20 05:04:11.877497 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.877229 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f1cf2be-098d-4985-bde4-081f4388c57b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-7jpfh\" (UID: \"5f1cf2be-098d-4985-bde4-081f4388c57b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:11.877497 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.877280 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhs8f\" (UniqueName: \"kubernetes.io/projected/5f1cf2be-098d-4985-bde4-081f4388c57b-kube-api-access-qhs8f\") pod \"openshift-state-metrics-79f6958cdf-7jpfh\" (UID: \"5f1cf2be-098d-4985-bde4-081f4388c57b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:11.877497 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.877312 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f1cf2be-098d-4985-bde4-081f4388c57b-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-7jpfh\" (UID: \"5f1cf2be-098d-4985-bde4-081f4388c57b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:11.877497 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.877347 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f1cf2be-098d-4985-bde4-081f4388c57b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-7jpfh\" (UID: \"5f1cf2be-098d-4985-bde4-081f4388c57b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:11.921633 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.921560 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978158 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978215 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978244 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f5274c4-78fe-4204-95e9-dc69ae66284f-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978267 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7eab4a6c-df4e-4a91-8214-64489829bf9f-root\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978290 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-tls\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978329 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978356 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7eab4a6c-df4e-4a91-8214-64489829bf9f-sys\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978391 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f1cf2be-098d-4985-bde4-081f4388c57b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-7jpfh\" (UID: \"5f1cf2be-098d-4985-bde4-081f4388c57b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978431 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978458 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhs8f\" (UniqueName: \"kubernetes.io/projected/5f1cf2be-098d-4985-bde4-081f4388c57b-kube-api-access-qhs8f\") pod \"openshift-state-metrics-79f6958cdf-7jpfh\" (UID: \"5f1cf2be-098d-4985-bde4-081f4388c57b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978491 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f1cf2be-098d-4985-bde4-081f4388c57b-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-7jpfh\" (UID: \"5f1cf2be-098d-4985-bde4-081f4388c57b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978528 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-accelerators-collector-config\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978551 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lm7x\" (UniqueName: \"kubernetes.io/projected/7eab4a6c-df4e-4a91-8214-64489829bf9f-kube-api-access-2lm7x\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978573 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-wtmp\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:11.979311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978600 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f1cf2be-098d-4985-bde4-081f4388c57b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-7jpfh\" (UID: \"5f1cf2be-098d-4985-bde4-081f4388c57b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:11.980806 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978628 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n9bs\" (UniqueName: \"kubernetes.io/projected/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-api-access-6n9bs\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:11.980806 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978660 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-textfile\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:11.980806 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978710 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eab4a6c-df4e-4a91-8214-64489829bf9f-metrics-client-ca\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:11.980806 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.978741 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5f5274c4-78fe-4204-95e9-dc69ae66284f-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:11.980806 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:04:11.979940 2576 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 20 05:04:11.980806 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:04:11.980063 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1cf2be-098d-4985-bde4-081f4388c57b-openshift-state-metrics-tls podName:5f1cf2be-098d-4985-bde4-081f4388c57b nodeName:}" failed. No retries permitted until 2026-07-20 05:04:12.480025084 +0000 UTC m=+295.526551971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5f1cf2be-098d-4985-bde4-081f4388c57b-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-7jpfh" (UID: "5f1cf2be-098d-4985-bde4-081f4388c57b") : secret "openshift-state-metrics-tls" not found Jul 20 05:04:11.980806 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.980395 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f1cf2be-098d-4985-bde4-081f4388c57b-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-7jpfh\" (UID: \"5f1cf2be-098d-4985-bde4-081f4388c57b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:11.982766 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.982723 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f1cf2be-098d-4985-bde4-081f4388c57b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-7jpfh\" (UID: \"5f1cf2be-098d-4985-bde4-081f4388c57b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:11.991329 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:11.991307 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhs8f\" (UniqueName: \"kubernetes.io/projected/5f1cf2be-098d-4985-bde4-081f4388c57b-kube-api-access-qhs8f\") pod \"openshift-state-metrics-79f6958cdf-7jpfh\" (UID: \"5f1cf2be-098d-4985-bde4-081f4388c57b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:12.079780 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.079735 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-textfile\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.080445 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.080315 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-textfile\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.081098 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.081052 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eab4a6c-df4e-4a91-8214-64489829bf9f-metrics-client-ca\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.081245 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.081205 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eab4a6c-df4e-4a91-8214-64489829bf9f-metrics-client-ca\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.081369 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.081354 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5f5274c4-78fe-4204-95e9-dc69ae66284f-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:12.081798 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.081783 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.082350 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.081730 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5f5274c4-78fe-4204-95e9-dc69ae66284f-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:12.082832 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:04:12.082516 2576 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 20 05:04:12.082832 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:04:12.082575 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-tls podName:5f5274c4-78fe-4204-95e9-dc69ae66284f nodeName:}" failed. No retries permitted until 2026-07-20 05:04:12.582558487 +0000 UTC m=+295.629085376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-qvbb9" (UID: "5f5274c4-78fe-4204-95e9-dc69ae66284f") : secret "kube-state-metrics-tls" not found Jul 20 05:04:12.083037 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.083020 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:12.083150 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.083138 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f5274c4-78fe-4204-95e9-dc69ae66284f-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:12.083831 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.083816 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7eab4a6c-df4e-4a91-8214-64489829bf9f-root\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.084042 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.084026 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-tls\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.084265 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.084250 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:12.085007 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.084991 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7eab4a6c-df4e-4a91-8214-64489829bf9f-sys\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.085211 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.085198 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:12.086001 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.085983 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-accelerators-collector-config\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.086576 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.086561 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lm7x\" (UniqueName: \"kubernetes.io/projected/7eab4a6c-df4e-4a91-8214-64489829bf9f-kube-api-access-2lm7x\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.087113 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.087095 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-wtmp\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.087403 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.087388 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n9bs\" (UniqueName: \"kubernetes.io/projected/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-api-access-6n9bs\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:12.087737 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.085889 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:12.087844 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.083983 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7eab4a6c-df4e-4a91-8214-64489829bf9f-root\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.087924 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.086510 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-accelerators-collector-config\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.088040 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.084919 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.088129 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.083778 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f5274c4-78fe-4204-95e9-dc69ae66284f-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:12.088208 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.087327 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-wtmp\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.088285 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.085129 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7eab4a6c-df4e-4a91-8214-64489829bf9f-sys\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.088353 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:04:12.084190 2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 20 05:04:12.088475 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:04:12.088466 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-tls podName:7eab4a6c-df4e-4a91-8214-64489829bf9f nodeName:}" failed. No retries permitted until 2026-07-20 05:04:12.588448613 +0000 UTC m=+295.634975511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-tls") pod "node-exporter-mh57v" (UID: "7eab4a6c-df4e-4a91-8214-64489829bf9f") : secret "node-exporter-tls" not found Jul 20 05:04:12.091171 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.091133 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:12.097523 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.097482 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n9bs\" (UniqueName: \"kubernetes.io/projected/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-api-access-6n9bs\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:12.106501 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.106450 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lm7x\" (UniqueName: \"kubernetes.io/projected/7eab4a6c-df4e-4a91-8214-64489829bf9f-kube-api-access-2lm7x\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.164451 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.164383 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:12.265558 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.265455 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:12.366098 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.365938 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:12.467445 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.467411 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:12.493034 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.492994 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f1cf2be-098d-4985-bde4-081f4388c57b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-7jpfh\" (UID: \"5f1cf2be-098d-4985-bde4-081f4388c57b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:12.495797 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.495773 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f1cf2be-098d-4985-bde4-081f4388c57b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-7jpfh\" (UID: \"5f1cf2be-098d-4985-bde4-081f4388c57b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:12.594201 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.594161 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:12.594670 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.594214 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-tls\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.594670 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:04:12.594303 2576 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 20 05:04:12.594670 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:04:12.594386 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-tls podName:5f5274c4-78fe-4204-95e9-dc69ae66284f nodeName:}" failed. No retries permitted until 2026-07-20 05:04:13.594363768 +0000 UTC m=+296.640890660 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-qvbb9" (UID: "5f5274c4-78fe-4204-95e9-dc69ae66284f") : secret "kube-state-metrics-tls" not found Jul 20 05:04:12.596742 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.596714 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7eab4a6c-df4e-4a91-8214-64489829bf9f-node-exporter-tls\") pod \"node-exporter-mh57v\" (UID: \"7eab4a6c-df4e-4a91-8214-64489829bf9f\") " pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.679163 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.679083 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:12.747873 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.747839 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" Jul 20 05:04:12.779975 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.779932 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mh57v" Jul 20 05:04:12.781637 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.781581 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:12.810707 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:04:12.810587 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eab4a6c_df4e_4a91_8214_64489829bf9f.slice/crio-d51016f2686cfd471ec9c7abf05d37c95aa79366dee57af4e479acd911f88805 WatchSource:0}: Error finding container d51016f2686cfd471ec9c7abf05d37c95aa79366dee57af4e479acd911f88805: Status 404 returned error can't find the container with id d51016f2686cfd471ec9c7abf05d37c95aa79366dee57af4e479acd911f88805 Jul 20 05:04:12.859267 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.859165 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:12.915902 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.915862 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh"] Jul 20 05:04:12.982415 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:12.982377 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:13.004556 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:04:13.004525 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1cf2be_098d_4985_bde4_081f4388c57b.slice/crio-5c09e53edccf5fb8f2138541b3b5e542bc9e8c3ee1e898a758400c25f4a1108c WatchSource:0}: Error finding container 5c09e53edccf5fb8f2138541b3b5e542bc9e8c3ee1e898a758400c25f4a1108c: Status 404 returned error can't find the container with id 5c09e53edccf5fb8f2138541b3b5e542bc9e8c3ee1e898a758400c25f4a1108c Jul 20 05:04:13.083018 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.082986 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:13.183854 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.183824 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:13.481783 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.481700 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:13.531814 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.531703 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:13.604457 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.604419 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:13.607266 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.607224 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f5274c4-78fe-4204-95e9-dc69ae66284f-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-qvbb9\" (UID: \"5f5274c4-78fe-4204-95e9-dc69ae66284f\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:13.662413 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.662025 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" Jul 20 05:04:13.672769 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.669976 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-z65tl" event={"ID":"91155ba9-0eab-446c-a27e-a2c2d7321584","Type":"ContainerStarted","Data":"ec0467915a2fc32eb264ae5885a6169325be5d293e8cbee4310c748ffb335246"} Jul 20 05:04:13.672769 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.670821 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-z65tl" Jul 20 05:04:13.673492 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.673462 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mh57v" event={"ID":"7eab4a6c-df4e-4a91-8214-64489829bf9f","Type":"ContainerStarted","Data":"d51016f2686cfd471ec9c7abf05d37c95aa79366dee57af4e479acd911f88805"} Jul 20 05:04:13.675662 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.675629 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cvb6r" event={"ID":"6a13f44c-570d-4fa6-96c6-bcdead54f7ec","Type":"ContainerStarted","Data":"03df4d6b840d85af4263b8f85617378a0d2611db16929d71453e78acd2ae352c"} Jul 20 05:04:13.675776 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.675675 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cvb6r" event={"ID":"6a13f44c-570d-4fa6-96c6-bcdead54f7ec","Type":"ContainerStarted","Data":"21076c13a212b166b0f0ae624cdfe3bec1ba2992b148df6fefba679528aedc94"} Jul 20 05:04:13.675931 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.675883 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cvb6r" Jul 20 05:04:13.678543 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.678517 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" event={"ID":"5f1cf2be-098d-4985-bde4-081f4388c57b","Type":"ContainerStarted","Data":"8b3da61274511aa4baa14a0e87bde1dd72fbef9efa1f80fb257bf8b6804fa162"} Jul 20 05:04:13.678652 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.678572 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" event={"ID":"5f1cf2be-098d-4985-bde4-081f4388c57b","Type":"ContainerStarted","Data":"8c31f216fb5b8ca4cda3d0bae2944fb55fa2181ee89029b24857b261e7bab190"} Jul 20 05:04:13.678652 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.678620 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" event={"ID":"5f1cf2be-098d-4985-bde4-081f4388c57b","Type":"ContainerStarted","Data":"5c09e53edccf5fb8f2138541b3b5e542bc9e8c3ee1e898a758400c25f4a1108c"} Jul 20 05:04:13.682916 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.682887 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-z65tl" Jul 20 05:04:13.710936 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.710870 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cvb6r" podStartSLOduration=252.119399243 podStartE2EDuration="4m21.710849339s" totalImagesPullingTime="9.591450096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 04:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:04:13.709000767 +0000 UTC m=+296.755527691" watchObservedRunningTime="2026-07-20 05:04:13.710849339 +0000 UTC m=+296.757376247" Jul 20 05:04:13.712479 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:13.712430 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-z65tl" podStartSLOduration=1.622596495 podStartE2EDuration="19.712414852s" totalImagesPullingTime="18.089818357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:03:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:04:13.691274767 +0000 UTC m=+296.737801685" watchObservedRunningTime="2026-07-20 05:04:13.712414852 +0000 UTC m=+296.758941761" Jul 20 05:04:14.116838 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:14.116707 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-qvbb9"] Jul 20 05:04:14.120141 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:04:14.120099 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5274c4_78fe_4204_95e9_dc69ae66284f.slice/crio-489cb6216f13caa750161f9164fbfa6ffeede7d9346ddcbcbdd7e26a5ccb4b22 WatchSource:0}: Error finding container 489cb6216f13caa750161f9164fbfa6ffeede7d9346ddcbcbdd7e26a5ccb4b22: Status 404 returned error can't find the container with id 489cb6216f13caa750161f9164fbfa6ffeede7d9346ddcbcbdd7e26a5ccb4b22 Jul 20 05:04:14.631692 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:14.631611 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:14.648321 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:04:14.648268 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-gnmjm" podUID="06a59c1d-4c72-423f-9e8d-27db0a919dba" Jul 20 05:04:14.686042 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:14.685824 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:14.689252 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:14.688282 2576 generic.go:356] "Generic (PLEG): container finished" podID="7eab4a6c-df4e-4a91-8214-64489829bf9f" containerID="ccc4c07d71e028ce6af0e8ebcb2c5b8373d117fa4f5f3ec3f8c017ad58308556" exitCode=0 Jul 20 05:04:14.689252 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:14.688408 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mh57v" event={"ID":"7eab4a6c-df4e-4a91-8214-64489829bf9f","Type":"ContainerDied","Data":"ccc4c07d71e028ce6af0e8ebcb2c5b8373d117fa4f5f3ec3f8c017ad58308556"} Jul 20 05:04:14.692583 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:14.692068 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" event={"ID":"5f5274c4-78fe-4204-95e9-dc69ae66284f","Type":"ContainerStarted","Data":"489cb6216f13caa750161f9164fbfa6ffeede7d9346ddcbcbdd7e26a5ccb4b22"} Jul 20 05:04:14.783546 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:14.783477 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:15.102378 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.102308 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:15.201536 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.201449 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:15.300341 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.300304 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:15.698993 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.698502 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" event={"ID":"5f1cf2be-098d-4985-bde4-081f4388c57b","Type":"ContainerStarted","Data":"e770f3084ed8b1441c2f7c444a8dd5b3c5cd753e02f0d04b634c6ed6689ceaf9"} Jul 20 05:04:15.703052 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.703016 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mh57v" event={"ID":"7eab4a6c-df4e-4a91-8214-64489829bf9f","Type":"ContainerStarted","Data":"8bb447d8b47414edea0b5883d32234275132d3cf584047def4530eb334133ca5"} Jul 20 05:04:15.703190 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.703075 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mh57v" event={"ID":"7eab4a6c-df4e-4a91-8214-64489829bf9f","Type":"ContainerStarted","Data":"c7a0fcf439e4a8298fab073d0a466cd175f0314427efce487ab6f25572bc5481"} Jul 20 05:04:15.718007 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.717927 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7jpfh" podStartSLOduration=3.094305632 podStartE2EDuration="4.71790998s" totalImagesPullingTime="1.623604348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:04:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:04:15.717218858 +0000 UTC m=+298.763745802" watchObservedRunningTime="2026-07-20 05:04:15.71790998 +0000 UTC m=+298.764436889" Jul 20 05:04:15.738315 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.738256 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mh57v" podStartSLOduration=3.470845362 podStartE2EDuration="4.738234063s" totalImagesPullingTime="1.174138856s" totalInitContainerRuntime="93.249845ms" isStatefulPod=true podCreationTimestamp="2026-07-20 05:04:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:04:15.737795807 +0000 UTC m=+298.784322758" watchObservedRunningTime="2026-07-20 05:04:15.738234063 +0000 UTC m=+298.784760970" Jul 20 05:04:15.821291 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.821248 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6ffb49c748-9lkrz"] Jul 20 05:04:15.853604 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.853565 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6ffb49c748-9lkrz"] Jul 20 05:04:15.853796 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.853752 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:15.862662 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.859446 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 20 05:04:15.862662 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.859770 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 20 05:04:15.862662 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.860076 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-elm0r385ho9qt\"" Jul 20 05:04:15.862662 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.860920 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 20 05:04:15.862662 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.861254 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vmzvl\"" Jul 20 05:04:15.862662 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.861467 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 20 05:04:15.862662 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.861685 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 20 05:04:15.931265 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.931222 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:15.931265 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.931274 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-grpc-tls\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:15.931481 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.931324 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhs7d\" (UniqueName: \"kubernetes.io/projected/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-kube-api-access-rhs7d\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:15.931481 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.931374 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:15.931481 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.931440 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-metrics-client-ca\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:15.931631 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.931489 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-tls\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:15.931631 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.931514 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:15.931631 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:15.931570 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.032045 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.032007 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.032045 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.032049 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-grpc-tls\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.032268 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.032081 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhs7d\" (UniqueName: \"kubernetes.io/projected/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-kube-api-access-rhs7d\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.032268 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.032111 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.032370 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.032295 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-metrics-client-ca\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.032370 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.032348 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-tls\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.032487 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.032384 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.033199 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.033136 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-metrics-client-ca\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.033199 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.033192 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.035307 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.035278 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-tls\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.035307 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.035289 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-grpc-tls\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.035972 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.035907 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.036308 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.036286 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.036415 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.036391 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.036483 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.036461 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.039753 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.039734 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhs7d\" (UniqueName: \"kubernetes.io/projected/9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0-kube-api-access-rhs7d\") pod \"thanos-querier-6ffb49c748-9lkrz\" (UID: \"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0\") " pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.170211 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.170171 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:16.215299 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.215265 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6dd68dc75-smzfb"] Jul 20 05:04:16.230554 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.230498 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dd68dc75-smzfb"] Jul 20 05:04:16.230702 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.230644 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.235004 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.233394 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 20 05:04:16.235004 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.233734 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-8tlw7\"" Jul 20 05:04:16.235004 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.234022 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 20 05:04:16.235004 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.234030 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-a0mbitc8900c5\"" Jul 20 05:04:16.235004 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.234043 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 20 05:04:16.235004 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.234303 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 20 05:04:16.336174 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.336143 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c7285c72-07f0-41f6-aa19-d40eeb35f04c-secret-metrics-server-client-certs\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.336353 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.336190 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c7285c72-07f0-41f6-aa19-d40eeb35f04c-audit-log\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.336353 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.336238 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c7285c72-07f0-41f6-aa19-d40eeb35f04c-metrics-server-audit-profiles\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.336353 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.336277 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c7285c72-07f0-41f6-aa19-d40eeb35f04c-secret-metrics-server-tls\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.336353 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.336326 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gln4s\" (UniqueName: \"kubernetes.io/projected/c7285c72-07f0-41f6-aa19-d40eeb35f04c-kube-api-access-gln4s\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.336579 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.336375 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7285c72-07f0-41f6-aa19-d40eeb35f04c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.336579 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.336498 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7285c72-07f0-41f6-aa19-d40eeb35f04c-client-ca-bundle\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.359451 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.359307 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6ffb49c748-9lkrz"] Jul 20 05:04:16.437336 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.437246 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gln4s\" (UniqueName: \"kubernetes.io/projected/c7285c72-07f0-41f6-aa19-d40eeb35f04c-kube-api-access-gln4s\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.437336 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.437297 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7285c72-07f0-41f6-aa19-d40eeb35f04c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.437336 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:04:16.437328 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f4b4bf0_c6bf_4027_b9e5_e81ea9f016e0.slice/crio-54993b8e46979124f18bc0900f9d045af4801ec3597979bd482711a30206036b WatchSource:0}: Error finding container 54993b8e46979124f18bc0900f9d045af4801ec3597979bd482711a30206036b: Status 404 returned error can't find the container with id 54993b8e46979124f18bc0900f9d045af4801ec3597979bd482711a30206036b Jul 20 05:04:16.437614 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.437373 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7285c72-07f0-41f6-aa19-d40eeb35f04c-client-ca-bundle\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.437614 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.437430 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c7285c72-07f0-41f6-aa19-d40eeb35f04c-secret-metrics-server-client-certs\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.437614 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.437459 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c7285c72-07f0-41f6-aa19-d40eeb35f04c-audit-log\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.437614 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.437503 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c7285c72-07f0-41f6-aa19-d40eeb35f04c-metrics-server-audit-profiles\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.437614 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.437528 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c7285c72-07f0-41f6-aa19-d40eeb35f04c-secret-metrics-server-tls\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.438267 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.437928 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c7285c72-07f0-41f6-aa19-d40eeb35f04c-audit-log\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.438267 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.438039 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7285c72-07f0-41f6-aa19-d40eeb35f04c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.439048 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.438977 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c7285c72-07f0-41f6-aa19-d40eeb35f04c-metrics-server-audit-profiles\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.441033 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.440949 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7285c72-07f0-41f6-aa19-d40eeb35f04c-client-ca-bundle\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.441033 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.440981 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c7285c72-07f0-41f6-aa19-d40eeb35f04c-secret-metrics-server-client-certs\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.442339 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.442318 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c7285c72-07f0-41f6-aa19-d40eeb35f04c-secret-metrics-server-tls\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.445676 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.445657 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gln4s\" (UniqueName: \"kubernetes.io/projected/c7285c72-07f0-41f6-aa19-d40eeb35f04c-kube-api-access-gln4s\") pod \"metrics-server-6dd68dc75-smzfb\" (UID: \"c7285c72-07f0-41f6-aa19-d40eeb35f04c\") " pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.547245 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.547215 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:16.594911 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.593368 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7899876bcf-zm2cl" Jul 20 05:04:16.594911 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.593394 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:16.677942 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.677354 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:16.715584 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.715552 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" event={"ID":"5f5274c4-78fe-4204-95e9-dc69ae66284f","Type":"ContainerStarted","Data":"8eb17df98025df91bac80ea19a492ebd5ad4b9ea1aa8df8cf79db94f5d237e71"} Jul 20 05:04:16.718440 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.718414 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" event={"ID":"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0","Type":"ContainerStarted","Data":"54993b8e46979124f18bc0900f9d045af4801ec3597979bd482711a30206036b"} Jul 20 05:04:16.787392 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.787361 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:16.796108 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.796021 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dd68dc75-smzfb"] Jul 20 05:04:16.811260 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:04:16.811218 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7285c72_07f0_41f6_aa19_d40eeb35f04c.slice/crio-64346cc161ded1e0a948e21d59887ca8891d9791bd9c6cda937ca3684dea9211 WatchSource:0}: Error finding container 64346cc161ded1e0a948e21d59887ca8891d9791bd9c6cda937ca3684dea9211: Status 404 returned error can't find the container with id 64346cc161ded1e0a948e21d59887ca8891d9791bd9c6cda937ca3684dea9211 Jul 20 05:04:16.887252 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.887212 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:16.991544 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:16.991427 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:17.613918 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:17.613862 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:04:17.616688 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:17.616660 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:04:17.624568 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:17.624542 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:04:17.627680 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:17.627660 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:04:17.641103 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:17.641080 2576 kubelet.go:1648] "Image garbage collection succeeded" Jul 20 05:04:17.728670 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:17.728434 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" event={"ID":"5f5274c4-78fe-4204-95e9-dc69ae66284f","Type":"ContainerStarted","Data":"3ccf4658c367f0577a5c6758e949c24216914ebd9878e7a4ed7cd9893e78a3c2"} Jul 20 05:04:17.728670 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:17.728550 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" event={"ID":"5f5274c4-78fe-4204-95e9-dc69ae66284f","Type":"ContainerStarted","Data":"e0d30cd819769901ec433bea5558241db801217a937f575cd0ccffed6553827e"} Jul 20 05:04:18.691063 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:17.732045 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" event={"ID":"c7285c72-07f0-41f6-aa19-d40eeb35f04c","Type":"ContainerStarted","Data":"64346cc161ded1e0a948e21d59887ca8891d9791bd9c6cda937ca3684dea9211"} Jul 20 05:04:18.691063 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:17.750324 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-qvbb9" podStartSLOduration=4.416111454 podStartE2EDuration="6.750302859s" totalImagesPullingTime="2.334191405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:04:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:04:17.748614024 +0000 UTC m=+300.795140931" watchObservedRunningTime="2026-07-20 05:04:17.750302859 +0000 UTC m=+300.796829766" Jul 20 05:04:18.691063 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:17.986839 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:18.691063 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:18.100464 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:18.691063 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:18.205122 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:20.327818 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:20.327791 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 05:04:20.750766 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:20.750722 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" event={"ID":"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0","Type":"ContainerStarted","Data":"657aa53a80077627ccbcb24712f64780f958a56b502d223304856bab88f4cc15"} Jul 20 05:04:20.750996 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:20.750818 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" event={"ID":"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0","Type":"ContainerStarted","Data":"b88ea947962692c7d32f4dc7e8d2f1efd2bea3c7f212d0edb6277879a22b8cfb"} Jul 20 05:04:20.750996 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:20.750859 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" event={"ID":"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0","Type":"ContainerStarted","Data":"03c34ae49d7529be8d3705204dd7ed47262a3e2c0e79711069672eede7db2f60"} Jul 20 05:04:20.752589 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:20.752557 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" event={"ID":"c7285c72-07f0-41f6-aa19-d40eeb35f04c","Type":"ContainerStarted","Data":"d1783188ea37a97db3a0f94a4f2ab10b48e1a803055a405e6490928b3c223c04"} Jul 20 05:04:20.768850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:20.768784 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" podStartSLOduration=1.6695065059999998 podStartE2EDuration="4.768761321s" totalImagesPullingTime="3.099254815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:04:20.767746979 +0000 UTC m=+303.814273886" watchObservedRunningTime="2026-07-20 05:04:20.768761321 +0000 UTC m=+303.815288229" Jul 20 05:04:21.097439 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.097356 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:21.195664 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.195619 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:21.366664 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.366581 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76cb99bc8b-9wxml"] Jul 20 05:04:21.388764 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.388714 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76cb99bc8b-9wxml"] Jul 20 05:04:21.388925 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.388828 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.391604 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.391574 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 20 05:04:21.391604 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.391598 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 20 05:04:21.391800 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.391711 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 20 05:04:21.392566 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.392545 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-25rk9\"" Jul 20 05:04:21.392566 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.392559 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 20 05:04:21.392759 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.392745 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 20 05:04:21.397680 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.397662 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 20 05:04:21.401913 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.401883 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:21.493088 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.493054 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-config\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.493279 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.493143 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-oauth-config\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.493279 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.493173 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmd5g\" (UniqueName: \"kubernetes.io/projected/15dbadbd-de41-4b61-9080-43961b5eb8a4-kube-api-access-mmd5g\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.493279 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.493217 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-service-ca\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.493436 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.493282 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-trusted-ca-bundle\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.493436 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.493373 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-serving-cert\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.493436 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.493403 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-oauth-serving-cert\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.502019 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.501982 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:21.594044 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.594001 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-oauth-config\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.594234 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.594053 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmd5g\" (UniqueName: \"kubernetes.io/projected/15dbadbd-de41-4b61-9080-43961b5eb8a4-kube-api-access-mmd5g\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.594234 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.594095 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-service-ca\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.594234 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.594122 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-trusted-ca-bundle\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.594234 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.594186 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-serving-cert\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.594234 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.594210 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-oauth-serving-cert\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.594661 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.594639 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-config\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.595200 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.595180 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-service-ca\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.595302 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.595209 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-oauth-serving-cert\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.595725 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.595700 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-config\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.596367 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.596344 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-trusted-ca-bundle\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.596917 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.596895 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-oauth-config\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.597186 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.597164 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-serving-cert\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.605585 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.605556 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmd5g\" (UniqueName: \"kubernetes.io/projected/15dbadbd-de41-4b61-9080-43961b5eb8a4-kube-api-access-mmd5g\") pod \"console-76cb99bc8b-9wxml\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.700712 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.700679 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:21.879605 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:21.879545 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76cb99bc8b-9wxml"] Jul 20 05:04:21.882848 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:04:21.882822 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15dbadbd_de41_4b61_9080_43961b5eb8a4.slice/crio-8b8e169b1aa55ef8a94ca1036dabb62aa12d7ec3c3eed1857e4058583911cdec WatchSource:0}: Error finding container 8b8e169b1aa55ef8a94ca1036dabb62aa12d7ec3c3eed1857e4058583911cdec: Status 404 returned error can't find the container with id 8b8e169b1aa55ef8a94ca1036dabb62aa12d7ec3c3eed1857e4058583911cdec Jul 20 05:04:22.766754 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:22.766707 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" event={"ID":"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0","Type":"ContainerStarted","Data":"4ac922f817dcb4492eedbb878427174a8c246b10178c13eba1b082c39fa5a492"} Jul 20 05:04:22.767268 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:22.766788 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" event={"ID":"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0","Type":"ContainerStarted","Data":"be481f7920846ddb7be2d8b1dc5f2d23ec0d699ce035e050a2847055859e735d"} Jul 20 05:04:22.767268 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:22.766822 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" event={"ID":"9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0","Type":"ContainerStarted","Data":"7d66dae2cdd325b37e279ea3b92a5847c09a1cfe54d6e0d743259d9a42896a4a"} Jul 20 05:04:22.767268 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:22.766894 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:22.768235 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:22.768208 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cb99bc8b-9wxml" event={"ID":"15dbadbd-de41-4b61-9080-43961b5eb8a4","Type":"ContainerStarted","Data":"8b8e169b1aa55ef8a94ca1036dabb62aa12d7ec3c3eed1857e4058583911cdec"} Jul 20 05:04:22.788577 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:22.788515 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" podStartSLOduration=2.960534546 podStartE2EDuration="7.788492893s" totalImagesPullingTime="4.827958347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:04:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:04:22.787163325 +0000 UTC m=+305.833690259" watchObservedRunningTime="2026-07-20 05:04:22.788492893 +0000 UTC m=+305.835019799" Jul 20 05:04:23.695361 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:23.695324 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cvb6r" Jul 20 05:04:25.784831 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:25.784792 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cb99bc8b-9wxml" event={"ID":"15dbadbd-de41-4b61-9080-43961b5eb8a4","Type":"ContainerStarted","Data":"d0ceff7cbe342fcd0004fa227ee20f31528052e429e277afa7f8b84528599eaa"} Jul 20 05:04:25.802276 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:25.802228 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76cb99bc8b-9wxml" podStartSLOduration=1.59181844 podStartE2EDuration="4.802208963s" totalImagesPullingTime="3.210390523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:04:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:04:25.80117006 +0000 UTC m=+308.847696994" watchObservedRunningTime="2026-07-20 05:04:25.802208963 +0000 UTC m=+308.848735869" Jul 20 05:04:26.646985 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:26.646922 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 05:04:26.871169 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:26.871134 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:26.922367 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:26.922287 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:28.780725 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:28.780699 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6ffb49c748-9lkrz" Jul 20 05:04:30.154096 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:30.154068 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:30.258098 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:30.258064 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:31.701292 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:31.701254 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:31.701654 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:31.701316 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:31.705732 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:31.705709 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:31.811536 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:31.811510 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:04:31.960197 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:31.960119 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:32.061464 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:32.061428 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:32.164732 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:32.164704 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:32.810529 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:32.810495 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:35.986277 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:35.986244 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:36.087021 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:36.086982 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:36.388792 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:36.388705 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:36.489000 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:36.488951 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:36.548142 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:36.548098 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:36.548311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:36.548189 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:41.815333 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:41.815297 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:41.912931 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:41.912901 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:50.886118 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:50.886085 2576 generic.go:356] "Generic (PLEG): container finished" podID="dced5a3f-757b-47aa-b3b2-2ada2072603f" containerID="33b11697f418411c5c3d397ad108c1a2808c73f6c6147c1843c88eb09678de8a" exitCode=0 Jul 20 05:04:50.886570 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:50.886160 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" event={"ID":"dced5a3f-757b-47aa-b3b2-2ada2072603f","Type":"ContainerDied","Data":"33b11697f418411c5c3d397ad108c1a2808c73f6c6147c1843c88eb09678de8a"} Jul 20 05:04:50.886570 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:50.886510 2576 scope.go:117] "RemoveContainer" containerID="33b11697f418411c5c3d397ad108c1a2808c73f6c6147c1843c88eb09678de8a" Jul 20 05:04:51.395880 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:51.395848 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:51.496172 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:51.496132 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:04:51.891510 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:51.891473 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-48w7b" event={"ID":"dced5a3f-757b-47aa-b3b2-2ada2072603f","Type":"ContainerStarted","Data":"4f01891e861a9edc40994ecb483614a96f43c629d3a25ad8c98d116c9b927205"} Jul 20 05:04:56.553184 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:56.553154 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:04:56.557129 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:04:56.557104 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6dd68dc75-smzfb" Jul 20 05:05:00.932942 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:00.932907 2576 generic.go:356] "Generic (PLEG): container finished" podID="9661ba55-2be6-499e-a1af-a5c0cb54f376" containerID="804a08922d4d12c961754251317f46b2adaaec152d0858ca49888613c757a9ce" exitCode=0 Jul 20 05:05:00.933362 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:00.932984 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-c9l2r" event={"ID":"9661ba55-2be6-499e-a1af-a5c0cb54f376","Type":"ContainerDied","Data":"804a08922d4d12c961754251317f46b2adaaec152d0858ca49888613c757a9ce"} Jul 20 05:05:00.933362 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:00.933329 2576 scope.go:117] "RemoveContainer" containerID="804a08922d4d12c961754251317f46b2adaaec152d0858ca49888613c757a9ce" Jul 20 05:05:01.938577 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:01.938543 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-c9l2r" event={"ID":"9661ba55-2be6-499e-a1af-a5c0cb54f376","Type":"ContainerStarted","Data":"35722c9ca32a98e99d9e0dd4d7e99ee451e5117248ab5e51053cf7968386a34c"} Jul 20 05:05:06.401531 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:06.401496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:06.498985 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:06.498933 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:20.971479 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:20.971398 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:21.024461 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:21.024428 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:27.879837 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:27.879799 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:27.934454 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:27.934417 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:29.876888 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:29.876846 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:29.934456 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:29.934418 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:31.535230 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:31.535190 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 05:05:31.537493 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:31.537474 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06a59c1d-4c72-423f-9e8d-27db0a919dba-metrics-certs\") pod \"network-metrics-daemon-gnmjm\" (UID: \"06a59c1d-4c72-423f-9e8d-27db0a919dba\") " pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 05:05:31.750753 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:31.750720 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z27sm\"" Jul 20 05:05:31.758694 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:31.758670 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gnmjm" Jul 20 05:05:31.892629 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:31.892604 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gnmjm"] Jul 20 05:05:31.895058 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:05:31.895030 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a59c1d_4c72_423f_9e8d_27db0a919dba.slice/crio-29487f1c5d57a5c67bfd7168faee0a9cc8fe145bae69ca684ef5edcfe0c9ab84 WatchSource:0}: Error finding container 29487f1c5d57a5c67bfd7168faee0a9cc8fe145bae69ca684ef5edcfe0c9ab84: Status 404 returned error can't find the container with id 29487f1c5d57a5c67bfd7168faee0a9cc8fe145bae69ca684ef5edcfe0c9ab84 Jul 20 05:05:32.057980 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:32.057928 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gnmjm" event={"ID":"06a59c1d-4c72-423f-9e8d-27db0a919dba","Type":"ContainerStarted","Data":"29487f1c5d57a5c67bfd7168faee0a9cc8fe145bae69ca684ef5edcfe0c9ab84"} Jul 20 05:05:33.069152 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:33.069066 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gnmjm" event={"ID":"06a59c1d-4c72-423f-9e8d-27db0a919dba","Type":"ContainerStarted","Data":"3f56fb22c4a14edf6f927b654d293c86514da2563e37d996d26a2ccca082606a"} Jul 20 05:05:33.069152 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:33.069140 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gnmjm" event={"ID":"06a59c1d-4c72-423f-9e8d-27db0a919dba","Type":"ContainerStarted","Data":"f3625acfbb773474f179a4a761f6a86d315d95abbe1d814b00832fc3c194e0f0"} Jul 20 05:05:33.084335 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:33.084289 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gnmjm" podStartSLOduration=374.183653149 podStartE2EDuration="6m15.084275329s" totalImagesPullingTime="900.62218ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 04:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:05:33.083925502 +0000 UTC m=+376.130452408" watchObservedRunningTime="2026-07-20 05:05:33.084275329 +0000 UTC m=+376.130802235" Jul 20 05:05:33.164918 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:33.164885 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:33.266585 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:33.266553 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:39.700200 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:39.700160 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:39.749495 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:39.749463 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:40.914315 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:40.914283 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76cb99bc8b-9wxml"] Jul 20 05:05:41.002786 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:41.002749 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:41.058806 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:41.058776 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:41.113768 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:41.113736 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:51.406396 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:51.406363 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:05:51.508235 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:05:51.508190 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:05.938263 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:05.938212 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76cb99bc8b-9wxml" podUID="15dbadbd-de41-4b61-9080-43961b5eb8a4" containerName="console" containerID="cri-o://d0ceff7cbe342fcd0004fa227ee20f31528052e429e277afa7f8b84528599eaa" gracePeriod=15 Jul 20 05:06:06.175683 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.175654 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:06:06.200766 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.200668 2576 generic.go:356] "Generic (PLEG): container finished" podID="15dbadbd-de41-4b61-9080-43961b5eb8a4" containerID="d0ceff7cbe342fcd0004fa227ee20f31528052e429e277afa7f8b84528599eaa" exitCode=0 Jul 20 05:06:06.200766 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.200713 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cb99bc8b-9wxml" Jul 20 05:06:06.201011 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.200752 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cb99bc8b-9wxml" event={"ID":"15dbadbd-de41-4b61-9080-43961b5eb8a4","Type":"ContainerDied","Data":"d0ceff7cbe342fcd0004fa227ee20f31528052e429e277afa7f8b84528599eaa"} Jul 20 05:06:06.201011 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.200821 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cb99bc8b-9wxml" event={"ID":"15dbadbd-de41-4b61-9080-43961b5eb8a4","Type":"ContainerDied","Data":"8b8e169b1aa55ef8a94ca1036dabb62aa12d7ec3c3eed1857e4058583911cdec"} Jul 20 05:06:06.201011 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.200851 2576 scope.go:117] "RemoveContainer" containerID="d0ceff7cbe342fcd0004fa227ee20f31528052e429e277afa7f8b84528599eaa" Jul 20 05:06:06.210773 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.210751 2576 scope.go:117] "RemoveContainer" containerID="d0ceff7cbe342fcd0004fa227ee20f31528052e429e277afa7f8b84528599eaa" Jul 20 05:06:06.211107 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:06:06.211086 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0ceff7cbe342fcd0004fa227ee20f31528052e429e277afa7f8b84528599eaa\": container with ID starting with d0ceff7cbe342fcd0004fa227ee20f31528052e429e277afa7f8b84528599eaa not found: ID does not exist" containerID="d0ceff7cbe342fcd0004fa227ee20f31528052e429e277afa7f8b84528599eaa" Jul 20 05:06:06.211169 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.211113 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0ceff7cbe342fcd0004fa227ee20f31528052e429e277afa7f8b84528599eaa"} err="failed to get container status \"d0ceff7cbe342fcd0004fa227ee20f31528052e429e277afa7f8b84528599eaa\": rpc error: code = NotFound desc = could not find container \"d0ceff7cbe342fcd0004fa227ee20f31528052e429e277afa7f8b84528599eaa\": container with ID starting with d0ceff7cbe342fcd0004fa227ee20f31528052e429e277afa7f8b84528599eaa not found: ID does not exist" Jul 20 05:06:06.233556 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.233525 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmd5g\" (UniqueName: \"kubernetes.io/projected/15dbadbd-de41-4b61-9080-43961b5eb8a4-kube-api-access-mmd5g\") pod \"15dbadbd-de41-4b61-9080-43961b5eb8a4\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " Jul 20 05:06:06.233696 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.233567 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-oauth-config\") pod \"15dbadbd-de41-4b61-9080-43961b5eb8a4\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " Jul 20 05:06:06.233696 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.233588 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-oauth-serving-cert\") pod \"15dbadbd-de41-4b61-9080-43961b5eb8a4\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " Jul 20 05:06:06.233696 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.233615 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-serving-cert\") pod \"15dbadbd-de41-4b61-9080-43961b5eb8a4\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " Jul 20 05:06:06.233696 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.233646 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-service-ca\") pod \"15dbadbd-de41-4b61-9080-43961b5eb8a4\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " Jul 20 05:06:06.233896 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.233722 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-trusted-ca-bundle\") pod \"15dbadbd-de41-4b61-9080-43961b5eb8a4\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " Jul 20 05:06:06.233896 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.233763 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-config\") pod \"15dbadbd-de41-4b61-9080-43961b5eb8a4\" (UID: \"15dbadbd-de41-4b61-9080-43961b5eb8a4\") " Jul 20 05:06:06.234187 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.234143 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-service-ca" (OuterVolumeSpecName: "service-ca") pod "15dbadbd-de41-4b61-9080-43961b5eb8a4" (UID: "15dbadbd-de41-4b61-9080-43961b5eb8a4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:06:06.234363 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.234204 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-config" (OuterVolumeSpecName: "console-config") pod "15dbadbd-de41-4b61-9080-43961b5eb8a4" (UID: "15dbadbd-de41-4b61-9080-43961b5eb8a4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:06:06.234363 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.234186 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "15dbadbd-de41-4b61-9080-43961b5eb8a4" (UID: "15dbadbd-de41-4b61-9080-43961b5eb8a4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:06:06.234363 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.234329 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "15dbadbd-de41-4b61-9080-43961b5eb8a4" (UID: "15dbadbd-de41-4b61-9080-43961b5eb8a4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:06:06.235905 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.235880 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15dbadbd-de41-4b61-9080-43961b5eb8a4-kube-api-access-mmd5g" (OuterVolumeSpecName: "kube-api-access-mmd5g") pod "15dbadbd-de41-4b61-9080-43961b5eb8a4" (UID: "15dbadbd-de41-4b61-9080-43961b5eb8a4"). InnerVolumeSpecName "kube-api-access-mmd5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:06:06.236157 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.236140 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "15dbadbd-de41-4b61-9080-43961b5eb8a4" (UID: "15dbadbd-de41-4b61-9080-43961b5eb8a4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 05:06:06.236234 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.236161 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "15dbadbd-de41-4b61-9080-43961b5eb8a4" (UID: "15dbadbd-de41-4b61-9080-43961b5eb8a4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 05:06:06.335280 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.335246 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-config\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:06:06.335280 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.335273 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmd5g\" (UniqueName: \"kubernetes.io/projected/15dbadbd-de41-4b61-9080-43961b5eb8a4-kube-api-access-mmd5g\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:06:06.335280 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.335283 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-oauth-config\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:06:06.335501 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.335292 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-oauth-serving-cert\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:06:06.335501 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.335302 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15dbadbd-de41-4b61-9080-43961b5eb8a4-console-serving-cert\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:06:06.335501 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.335312 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-service-ca\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:06:06.335501 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.335320 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15dbadbd-de41-4b61-9080-43961b5eb8a4-trusted-ca-bundle\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:06:06.521190 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.521152 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76cb99bc8b-9wxml"] Jul 20 05:06:06.523401 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:06.523371 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76cb99bc8b-9wxml"] Jul 20 05:06:07.657203 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:07.657167 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15dbadbd-de41-4b61-9080-43961b5eb8a4" path="/var/lib/kubelet/pods/15dbadbd-de41-4b61-9080-43961b5eb8a4/volumes" Jul 20 05:06:19.043580 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.043543 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:19.143611 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.143576 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:19.612632 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.612596 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp"] Jul 20 05:06:19.612943 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.612930 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15dbadbd-de41-4b61-9080-43961b5eb8a4" containerName="console" Jul 20 05:06:19.613013 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.612944 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="15dbadbd-de41-4b61-9080-43961b5eb8a4" containerName="console" Jul 20 05:06:19.613013 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.613011 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="15dbadbd-de41-4b61-9080-43961b5eb8a4" containerName="console" Jul 20 05:06:19.617232 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.617212 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" Jul 20 05:06:19.623211 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.623185 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gmkgv\"" Jul 20 05:06:19.623447 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.623417 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 05:06:19.623770 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.623454 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 05:06:19.625810 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.625785 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp"] Jul 20 05:06:19.746843 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.746803 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fcec1e1-44ce-48ca-a135-6574d5a7559e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp\" (UID: \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" Jul 20 05:06:19.747074 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.746930 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fcec1e1-44ce-48ca-a135-6574d5a7559e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp\" (UID: \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" Jul 20 05:06:19.747074 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.747019 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmpxd\" (UniqueName: \"kubernetes.io/projected/8fcec1e1-44ce-48ca-a135-6574d5a7559e-kube-api-access-xmpxd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp\" (UID: \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" Jul 20 05:06:19.847663 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.847624 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fcec1e1-44ce-48ca-a135-6574d5a7559e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp\" (UID: \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" Jul 20 05:06:19.847824 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.847684 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmpxd\" (UniqueName: \"kubernetes.io/projected/8fcec1e1-44ce-48ca-a135-6574d5a7559e-kube-api-access-xmpxd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp\" (UID: \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" Jul 20 05:06:19.847824 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.847713 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fcec1e1-44ce-48ca-a135-6574d5a7559e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp\" (UID: \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" Jul 20 05:06:19.848068 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.848046 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fcec1e1-44ce-48ca-a135-6574d5a7559e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp\" (UID: \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" Jul 20 05:06:19.848141 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.848120 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fcec1e1-44ce-48ca-a135-6574d5a7559e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp\" (UID: \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" Jul 20 05:06:19.857245 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.857218 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmpxd\" (UniqueName: \"kubernetes.io/projected/8fcec1e1-44ce-48ca-a135-6574d5a7559e-kube-api-access-xmpxd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp\" (UID: \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" Jul 20 05:06:19.933168 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:19.933083 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" Jul 20 05:06:20.067096 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:20.067013 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp"] Jul 20 05:06:20.069820 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:06:20.069789 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fcec1e1_44ce_48ca_a135_6574d5a7559e.slice/crio-6c971afd5e18e348184d4ef1b4a5cf1a940c599aa5d87627df5288e06f7ba9df WatchSource:0}: Error finding container 6c971afd5e18e348184d4ef1b4a5cf1a940c599aa5d87627df5288e06f7ba9df: Status 404 returned error can't find the container with id 6c971afd5e18e348184d4ef1b4a5cf1a940c599aa5d87627df5288e06f7ba9df Jul 20 05:06:20.260793 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:20.260754 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" event={"ID":"8fcec1e1-44ce-48ca-a135-6574d5a7559e","Type":"ContainerStarted","Data":"6c971afd5e18e348184d4ef1b4a5cf1a940c599aa5d87627df5288e06f7ba9df"} Jul 20 05:06:27.294547 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:27.294509 2576 generic.go:356] "Generic (PLEG): container finished" podID="8fcec1e1-44ce-48ca-a135-6574d5a7559e" containerID="6e0c3371c010cd10bbedb6eb0bcba637ce4bcabf0b78e9783c7aadf5a981c69e" exitCode=0 Jul 20 05:06:27.295029 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:27.294597 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" event={"ID":"8fcec1e1-44ce-48ca-a135-6574d5a7559e","Type":"ContainerDied","Data":"6e0c3371c010cd10bbedb6eb0bcba637ce4bcabf0b78e9783c7aadf5a981c69e"} Jul 20 05:06:29.304406 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:29.304375 2576 generic.go:356] "Generic (PLEG): container finished" podID="8fcec1e1-44ce-48ca-a135-6574d5a7559e" containerID="46078eb1866013cd5cbdd823114c62f7a40149b57cf4c6d1689e6a85c2da3c8a" exitCode=0 Jul 20 05:06:29.304854 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:29.304467 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" event={"ID":"8fcec1e1-44ce-48ca-a135-6574d5a7559e","Type":"ContainerDied","Data":"46078eb1866013cd5cbdd823114c62f7a40149b57cf4c6d1689e6a85c2da3c8a"} Jul 20 05:06:32.010666 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:32.010024 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:36.337331 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:36.337280 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" event={"ID":"8fcec1e1-44ce-48ca-a135-6574d5a7559e","Type":"ContainerStarted","Data":"b2500b8c9db9b7dd65c774f2edf07449316f580850cad401337a3413e579a3e3"} Jul 20 05:06:36.354172 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:36.354125 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" podStartSLOduration=2.393613983 podStartE2EDuration="17.354108684s" totalImagesPullingTime="14.960494701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:06:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:06:36.352858189 +0000 UTC m=+439.399385096" watchObservedRunningTime="2026-07-20 05:06:36.354108684 +0000 UTC m=+439.400635589" Jul 20 05:06:37.343906 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:37.343872 2576 generic.go:356] "Generic (PLEG): container finished" podID="8fcec1e1-44ce-48ca-a135-6574d5a7559e" containerID="b2500b8c9db9b7dd65c774f2edf07449316f580850cad401337a3413e579a3e3" exitCode=0 Jul 20 05:06:37.344302 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:37.343978 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" event={"ID":"8fcec1e1-44ce-48ca-a135-6574d5a7559e","Type":"ContainerDied","Data":"b2500b8c9db9b7dd65c774f2edf07449316f580850cad401337a3413e579a3e3"} Jul 20 05:06:38.481353 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:38.481324 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" Jul 20 05:06:38.621514 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:38.621418 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fcec1e1-44ce-48ca-a135-6574d5a7559e-bundle\") pod \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\" (UID: \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\") " Jul 20 05:06:38.621514 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:38.621493 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fcec1e1-44ce-48ca-a135-6574d5a7559e-util\") pod \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\" (UID: \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\") " Jul 20 05:06:38.621697 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:38.621569 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmpxd\" (UniqueName: \"kubernetes.io/projected/8fcec1e1-44ce-48ca-a135-6574d5a7559e-kube-api-access-xmpxd\") pod \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\" (UID: \"8fcec1e1-44ce-48ca-a135-6574d5a7559e\") " Jul 20 05:06:38.622188 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:38.622157 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fcec1e1-44ce-48ca-a135-6574d5a7559e-bundle" (OuterVolumeSpecName: "bundle") pod "8fcec1e1-44ce-48ca-a135-6574d5a7559e" (UID: "8fcec1e1-44ce-48ca-a135-6574d5a7559e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:06:38.623708 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:38.623677 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fcec1e1-44ce-48ca-a135-6574d5a7559e-kube-api-access-xmpxd" (OuterVolumeSpecName: "kube-api-access-xmpxd") pod "8fcec1e1-44ce-48ca-a135-6574d5a7559e" (UID: "8fcec1e1-44ce-48ca-a135-6574d5a7559e"). InnerVolumeSpecName "kube-api-access-xmpxd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:06:38.625562 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:38.625538 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fcec1e1-44ce-48ca-a135-6574d5a7559e-util" (OuterVolumeSpecName: "util") pod "8fcec1e1-44ce-48ca-a135-6574d5a7559e" (UID: "8fcec1e1-44ce-48ca-a135-6574d5a7559e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:06:38.723292 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:38.723256 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fcec1e1-44ce-48ca-a135-6574d5a7559e-bundle\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:06:38.723292 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:38.723285 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fcec1e1-44ce-48ca-a135-6574d5a7559e-util\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:06:38.723292 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:38.723296 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmpxd\" (UniqueName: \"kubernetes.io/projected/8fcec1e1-44ce-48ca-a135-6574d5a7559e-kube-api-access-xmpxd\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:06:39.354355 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:39.354273 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" event={"ID":"8fcec1e1-44ce-48ca-a135-6574d5a7559e","Type":"ContainerDied","Data":"6c971afd5e18e348184d4ef1b4a5cf1a940c599aa5d87627df5288e06f7ba9df"} Jul 20 05:06:39.354355 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:39.354299 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m9jp" Jul 20 05:06:39.354355 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:39.354321 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c971afd5e18e348184d4ef1b4a5cf1a940c599aa5d87627df5288e06f7ba9df" Jul 20 05:06:40.352460 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:40.352428 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:41.755172 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:41.755140 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:42.295986 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.295934 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v"] Jul 20 05:06:42.296298 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.296286 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fcec1e1-44ce-48ca-a135-6574d5a7559e" containerName="util" Jul 20 05:06:42.296345 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.296300 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fcec1e1-44ce-48ca-a135-6574d5a7559e" containerName="util" Jul 20 05:06:42.296345 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.296310 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fcec1e1-44ce-48ca-a135-6574d5a7559e" containerName="extract" Jul 20 05:06:42.296345 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.296316 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fcec1e1-44ce-48ca-a135-6574d5a7559e" containerName="extract" Jul 20 05:06:42.296345 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.296344 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fcec1e1-44ce-48ca-a135-6574d5a7559e" containerName="pull" Jul 20 05:06:42.296457 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.296351 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fcec1e1-44ce-48ca-a135-6574d5a7559e" containerName="pull" Jul 20 05:06:42.296457 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.296427 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fcec1e1-44ce-48ca-a135-6574d5a7559e" containerName="extract" Jul 20 05:06:42.299403 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.299387 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v" Jul 20 05:06:42.301915 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.301890 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 20 05:06:42.302110 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.301935 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 20 05:06:42.302110 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.302023 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-rvb87\"" Jul 20 05:06:42.314135 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.314101 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v"] Jul 20 05:06:42.453443 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.453386 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f0c9313-a1da-404a-acfa-08a46c1e1bb7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-82t2v\" (UID: \"8f0c9313-a1da-404a-acfa-08a46c1e1bb7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v" Jul 20 05:06:42.453625 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.453462 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p62zf\" (UniqueName: \"kubernetes.io/projected/8f0c9313-a1da-404a-acfa-08a46c1e1bb7-kube-api-access-p62zf\") pod \"cert-manager-operator-controller-manager-55c444bf86-82t2v\" (UID: \"8f0c9313-a1da-404a-acfa-08a46c1e1bb7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v" Jul 20 05:06:42.554899 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.554806 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f0c9313-a1da-404a-acfa-08a46c1e1bb7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-82t2v\" (UID: \"8f0c9313-a1da-404a-acfa-08a46c1e1bb7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v" Jul 20 05:06:42.554899 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.554891 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p62zf\" (UniqueName: \"kubernetes.io/projected/8f0c9313-a1da-404a-acfa-08a46c1e1bb7-kube-api-access-p62zf\") pod \"cert-manager-operator-controller-manager-55c444bf86-82t2v\" (UID: \"8f0c9313-a1da-404a-acfa-08a46c1e1bb7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v" Jul 20 05:06:42.555249 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.555231 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f0c9313-a1da-404a-acfa-08a46c1e1bb7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-82t2v\" (UID: \"8f0c9313-a1da-404a-acfa-08a46c1e1bb7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v" Jul 20 05:06:42.563267 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.563243 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p62zf\" (UniqueName: \"kubernetes.io/projected/8f0c9313-a1da-404a-acfa-08a46c1e1bb7-kube-api-access-p62zf\") pod \"cert-manager-operator-controller-manager-55c444bf86-82t2v\" (UID: \"8f0c9313-a1da-404a-acfa-08a46c1e1bb7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v" Jul 20 05:06:42.609251 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.609216 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v" Jul 20 05:06:42.757492 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:42.757465 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v"] Jul 20 05:06:42.760218 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:06:42.760187 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f0c9313_a1da_404a_acfa_08a46c1e1bb7.slice/crio-08d8a11e1177ff9d5df4d82666c92aeaaa65863084f4e3f9344b3f90a977e743 WatchSource:0}: Error finding container 08d8a11e1177ff9d5df4d82666c92aeaaa65863084f4e3f9344b3f90a977e743: Status 404 returned error can't find the container with id 08d8a11e1177ff9d5df4d82666c92aeaaa65863084f4e3f9344b3f90a977e743 Jul 20 05:06:43.376979 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:43.376923 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v" event={"ID":"8f0c9313-a1da-404a-acfa-08a46c1e1bb7","Type":"ContainerStarted","Data":"08d8a11e1177ff9d5df4d82666c92aeaaa65863084f4e3f9344b3f90a977e743"} Jul 20 05:06:43.968620 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:43.968583 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:44.069127 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:44.069090 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:45.389589 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:45.389550 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v" event={"ID":"8f0c9313-a1da-404a-acfa-08a46c1e1bb7","Type":"ContainerStarted","Data":"81c7028b07c56aec1915f83fb5f22b60f7f7cc556948e5731391a2b841d2c48a"} Jul 20 05:06:45.414828 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:45.414762 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82t2v" podStartSLOduration=1.282510726 podStartE2EDuration="3.414742253s" totalImagesPullingTime="2.132231527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:06:45.413356952 +0000 UTC m=+448.459883859" watchObservedRunningTime="2026-07-20 05:06:45.414742253 +0000 UTC m=+448.461269166" Jul 20 05:06:45.785003 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:45.784971 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:45.888732 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:45.888701 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:46.590881 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.590851 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v"] Jul 20 05:06:46.609917 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.609878 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v"] Jul 20 05:06:46.610083 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.610002 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" Jul 20 05:06:46.612842 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.612807 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 05:06:46.613000 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.612812 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 05:06:46.613000 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.612859 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gmkgv\"" Jul 20 05:06:46.791278 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.791237 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1275181-1d9b-4d70-bea5-91d2360109c2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v\" (UID: \"e1275181-1d9b-4d70-bea5-91d2360109c2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" Jul 20 05:06:46.791278 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.791278 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1275181-1d9b-4d70-bea5-91d2360109c2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v\" (UID: \"e1275181-1d9b-4d70-bea5-91d2360109c2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" Jul 20 05:06:46.791484 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.791299 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnmwt\" (UniqueName: \"kubernetes.io/projected/e1275181-1d9b-4d70-bea5-91d2360109c2-kube-api-access-wnmwt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v\" (UID: \"e1275181-1d9b-4d70-bea5-91d2360109c2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" Jul 20 05:06:46.892599 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.892517 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1275181-1d9b-4d70-bea5-91d2360109c2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v\" (UID: \"e1275181-1d9b-4d70-bea5-91d2360109c2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" Jul 20 05:06:46.892599 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.892557 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1275181-1d9b-4d70-bea5-91d2360109c2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v\" (UID: \"e1275181-1d9b-4d70-bea5-91d2360109c2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" Jul 20 05:06:46.892599 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.892587 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnmwt\" (UniqueName: \"kubernetes.io/projected/e1275181-1d9b-4d70-bea5-91d2360109c2-kube-api-access-wnmwt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v\" (UID: \"e1275181-1d9b-4d70-bea5-91d2360109c2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" Jul 20 05:06:46.892929 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.892910 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1275181-1d9b-4d70-bea5-91d2360109c2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v\" (UID: \"e1275181-1d9b-4d70-bea5-91d2360109c2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" Jul 20 05:06:46.893011 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.892990 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1275181-1d9b-4d70-bea5-91d2360109c2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v\" (UID: \"e1275181-1d9b-4d70-bea5-91d2360109c2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" Jul 20 05:06:46.900538 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.900514 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnmwt\" (UniqueName: \"kubernetes.io/projected/e1275181-1d9b-4d70-bea5-91d2360109c2-kube-api-access-wnmwt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v\" (UID: \"e1275181-1d9b-4d70-bea5-91d2360109c2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" Jul 20 05:06:46.920434 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:46.920401 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" Jul 20 05:06:47.076339 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:47.076238 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v"] Jul 20 05:06:47.079231 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:06:47.079205 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1275181_1d9b_4d70_bea5_91d2360109c2.slice/crio-54594da2b10a50f1ee74e3b3c5d5252be4c42d34302d9268bddfbb114abb4f74 WatchSource:0}: Error finding container 54594da2b10a50f1ee74e3b3c5d5252be4c42d34302d9268bddfbb114abb4f74: Status 404 returned error can't find the container with id 54594da2b10a50f1ee74e3b3c5d5252be4c42d34302d9268bddfbb114abb4f74 Jul 20 05:06:47.399729 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:47.399651 2576 generic.go:356] "Generic (PLEG): container finished" podID="e1275181-1d9b-4d70-bea5-91d2360109c2" containerID="f0fec01288aae85b5a9883dad7024c785008dd91afc06c9113b57d16e2841397" exitCode=0 Jul 20 05:06:47.399729 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:47.399687 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" event={"ID":"e1275181-1d9b-4d70-bea5-91d2360109c2","Type":"ContainerDied","Data":"f0fec01288aae85b5a9883dad7024c785008dd91afc06c9113b57d16e2841397"} Jul 20 05:06:47.399902 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:47.399743 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" event={"ID":"e1275181-1d9b-4d70-bea5-91d2360109c2","Type":"ContainerStarted","Data":"54594da2b10a50f1ee74e3b3c5d5252be4c42d34302d9268bddfbb114abb4f74"} Jul 20 05:06:48.194764 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:48.194676 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:48.618033 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:48.617967 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:48.787187 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:48.787156 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:50.416497 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:50.416411 2576 generic.go:356] "Generic (PLEG): container finished" podID="e1275181-1d9b-4d70-bea5-91d2360109c2" containerID="cbc82b6e7f96f0818d935141d1495a46b58bd17c6c9a5bf87a480b153ec5ec43" exitCode=0 Jul 20 05:06:50.416857 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:50.416499 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" event={"ID":"e1275181-1d9b-4d70-bea5-91d2360109c2","Type":"ContainerDied","Data":"cbc82b6e7f96f0818d935141d1495a46b58bd17c6c9a5bf87a480b153ec5ec43"} Jul 20 05:06:50.972321 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:50.972290 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:51.030050 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:51.030021 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:51.422840 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:51.422802 2576 generic.go:356] "Generic (PLEG): container finished" podID="e1275181-1d9b-4d70-bea5-91d2360109c2" containerID="8180e8db3b431d360b365d48bce11c8c46df647d454164e03de1f0e3af362764" exitCode=0 Jul 20 05:06:51.423255 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:51.422870 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" event={"ID":"e1275181-1d9b-4d70-bea5-91d2360109c2","Type":"ContainerDied","Data":"8180e8db3b431d360b365d48bce11c8c46df647d454164e03de1f0e3af362764"} Jul 20 05:06:52.560362 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:52.560335 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" Jul 20 05:06:52.649149 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:52.649109 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnmwt\" (UniqueName: \"kubernetes.io/projected/e1275181-1d9b-4d70-bea5-91d2360109c2-kube-api-access-wnmwt\") pod \"e1275181-1d9b-4d70-bea5-91d2360109c2\" (UID: \"e1275181-1d9b-4d70-bea5-91d2360109c2\") " Jul 20 05:06:52.649344 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:52.649196 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1275181-1d9b-4d70-bea5-91d2360109c2-bundle\") pod \"e1275181-1d9b-4d70-bea5-91d2360109c2\" (UID: \"e1275181-1d9b-4d70-bea5-91d2360109c2\") " Jul 20 05:06:52.649344 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:52.649291 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1275181-1d9b-4d70-bea5-91d2360109c2-util\") pod \"e1275181-1d9b-4d70-bea5-91d2360109c2\" (UID: \"e1275181-1d9b-4d70-bea5-91d2360109c2\") " Jul 20 05:06:52.649679 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:52.649648 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1275181-1d9b-4d70-bea5-91d2360109c2-bundle" (OuterVolumeSpecName: "bundle") pod "e1275181-1d9b-4d70-bea5-91d2360109c2" (UID: "e1275181-1d9b-4d70-bea5-91d2360109c2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:06:52.651249 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:52.651226 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1275181-1d9b-4d70-bea5-91d2360109c2-kube-api-access-wnmwt" (OuterVolumeSpecName: "kube-api-access-wnmwt") pod "e1275181-1d9b-4d70-bea5-91d2360109c2" (UID: "e1275181-1d9b-4d70-bea5-91d2360109c2"). InnerVolumeSpecName "kube-api-access-wnmwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:06:52.654161 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:52.654139 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1275181-1d9b-4d70-bea5-91d2360109c2-util" (OuterVolumeSpecName: "util") pod "e1275181-1d9b-4d70-bea5-91d2360109c2" (UID: "e1275181-1d9b-4d70-bea5-91d2360109c2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:06:52.750390 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:52.750358 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnmwt\" (UniqueName: \"kubernetes.io/projected/e1275181-1d9b-4d70-bea5-91d2360109c2-kube-api-access-wnmwt\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:06:52.750390 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:52.750390 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1275181-1d9b-4d70-bea5-91d2360109c2-bundle\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:06:52.750584 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:52.750400 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1275181-1d9b-4d70-bea5-91d2360109c2-util\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:06:53.433592 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:53.433555 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" event={"ID":"e1275181-1d9b-4d70-bea5-91d2360109c2","Type":"ContainerDied","Data":"54594da2b10a50f1ee74e3b3c5d5252be4c42d34302d9268bddfbb114abb4f74"} Jul 20 05:06:53.433592 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:53.433581 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4fk6v" Jul 20 05:06:53.433592 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:53.433601 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54594da2b10a50f1ee74e3b3c5d5252be4c42d34302d9268bddfbb114abb4f74" Jul 20 05:06:55.794353 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:55.794317 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:56.401179 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.401141 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:56.422894 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.422860 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rgljv"] Jul 20 05:06:56.423214 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.423199 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1275181-1d9b-4d70-bea5-91d2360109c2" containerName="pull" Jul 20 05:06:56.423257 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.423216 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1275181-1d9b-4d70-bea5-91d2360109c2" containerName="pull" Jul 20 05:06:56.423257 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.423230 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1275181-1d9b-4d70-bea5-91d2360109c2" containerName="extract" Jul 20 05:06:56.423257 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.423235 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1275181-1d9b-4d70-bea5-91d2360109c2" containerName="extract" Jul 20 05:06:56.423257 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.423242 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1275181-1d9b-4d70-bea5-91d2360109c2" containerName="util" Jul 20 05:06:56.423257 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.423247 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1275181-1d9b-4d70-bea5-91d2360109c2" containerName="util" Jul 20 05:06:56.423401 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.423305 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1275181-1d9b-4d70-bea5-91d2360109c2" containerName="extract" Jul 20 05:06:56.427445 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.427431 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rgljv" Jul 20 05:06:56.430808 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.430788 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 20 05:06:56.430942 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.430789 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ldsqj\"" Jul 20 05:06:56.430942 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.430792 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 20 05:06:56.436171 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.436148 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rgljv"] Jul 20 05:06:56.460059 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.460020 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:56.483542 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.483505 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bpp2\" (UniqueName: \"kubernetes.io/projected/d207af5c-9dc8-41f2-96f6-d8ab032607d9-kube-api-access-7bpp2\") pod \"cert-manager-55b4d954d5-rgljv\" (UID: \"d207af5c-9dc8-41f2-96f6-d8ab032607d9\") " pod="cert-manager/cert-manager-55b4d954d5-rgljv" Jul 20 05:06:56.483542 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.483545 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d207af5c-9dc8-41f2-96f6-d8ab032607d9-bound-sa-token\") pod \"cert-manager-55b4d954d5-rgljv\" (UID: \"d207af5c-9dc8-41f2-96f6-d8ab032607d9\") " pod="cert-manager/cert-manager-55b4d954d5-rgljv" Jul 20 05:06:56.584127 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.584081 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bpp2\" (UniqueName: \"kubernetes.io/projected/d207af5c-9dc8-41f2-96f6-d8ab032607d9-kube-api-access-7bpp2\") pod \"cert-manager-55b4d954d5-rgljv\" (UID: \"d207af5c-9dc8-41f2-96f6-d8ab032607d9\") " pod="cert-manager/cert-manager-55b4d954d5-rgljv" Jul 20 05:06:56.584127 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.584137 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d207af5c-9dc8-41f2-96f6-d8ab032607d9-bound-sa-token\") pod \"cert-manager-55b4d954d5-rgljv\" (UID: \"d207af5c-9dc8-41f2-96f6-d8ab032607d9\") " pod="cert-manager/cert-manager-55b4d954d5-rgljv" Jul 20 05:06:56.592591 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.592554 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d207af5c-9dc8-41f2-96f6-d8ab032607d9-bound-sa-token\") pod \"cert-manager-55b4d954d5-rgljv\" (UID: \"d207af5c-9dc8-41f2-96f6-d8ab032607d9\") " pod="cert-manager/cert-manager-55b4d954d5-rgljv" Jul 20 05:06:56.592707 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.592671 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bpp2\" (UniqueName: \"kubernetes.io/projected/d207af5c-9dc8-41f2-96f6-d8ab032607d9-kube-api-access-7bpp2\") pod \"cert-manager-55b4d954d5-rgljv\" (UID: \"d207af5c-9dc8-41f2-96f6-d8ab032607d9\") " pod="cert-manager/cert-manager-55b4d954d5-rgljv" Jul 20 05:06:56.749950 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.749916 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rgljv" Jul 20 05:06:56.886497 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:56.886470 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rgljv"] Jul 20 05:06:56.889100 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:06:56.889073 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd207af5c_9dc8_41f2_96f6_d8ab032607d9.slice/crio-5a7c5bd7e0cb5cae132b8d531f0333c877339737f633c8dec7ad45e2bdc43256 WatchSource:0}: Error finding container 5a7c5bd7e0cb5cae132b8d531f0333c877339737f633c8dec7ad45e2bdc43256: Status 404 returned error can't find the container with id 5a7c5bd7e0cb5cae132b8d531f0333c877339737f633c8dec7ad45e2bdc43256 Jul 20 05:06:57.374838 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:57.374802 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:06:57.454445 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:57.454402 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rgljv" event={"ID":"d207af5c-9dc8-41f2-96f6-d8ab032607d9","Type":"ContainerStarted","Data":"5a7c5bd7e0cb5cae132b8d531f0333c877339737f633c8dec7ad45e2bdc43256"} Jul 20 05:06:57.472285 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:06:57.472255 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:00.472675 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:00.472632 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rgljv" event={"ID":"d207af5c-9dc8-41f2-96f6-d8ab032607d9","Type":"ContainerStarted","Data":"40d68d836a4365ebe2e1563978334355c262f824abed1c3e9ebca8dd06ccd662"} Jul 20 05:07:00.488469 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:00.488410 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rgljv" podStartSLOduration=1.9828602640000002 podStartE2EDuration="4.488393258s" totalImagesPullingTime="2.505532994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:07:00.487625108 +0000 UTC m=+463.534152026" watchObservedRunningTime="2026-07-20 05:07:00.488393258 +0000 UTC m=+463.534920163" Jul 20 05:07:24.949546 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:24.949508 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:25.050192 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:25.050158 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:26.969041 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:26.969010 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:27.241526 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.241486 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g"] Jul 20 05:07:27.249483 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.249446 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" Jul 20 05:07:27.251588 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.251561 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g"] Jul 20 05:07:27.251906 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.251889 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gmkgv\"" Jul 20 05:07:27.252007 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.251904 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 05:07:27.252007 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.251908 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 05:07:27.354803 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.354762 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/795ef04e-05bf-4d4f-b383-13903e91464d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g\" (UID: \"795ef04e-05bf-4d4f-b383-13903e91464d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" Jul 20 05:07:27.355009 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.354813 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z87b\" (UniqueName: \"kubernetes.io/projected/795ef04e-05bf-4d4f-b383-13903e91464d-kube-api-access-8z87b\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g\" (UID: \"795ef04e-05bf-4d4f-b383-13903e91464d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" Jul 20 05:07:27.355009 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.354892 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/795ef04e-05bf-4d4f-b383-13903e91464d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g\" (UID: \"795ef04e-05bf-4d4f-b383-13903e91464d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" Jul 20 05:07:27.456303 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.456271 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/795ef04e-05bf-4d4f-b383-13903e91464d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g\" (UID: \"795ef04e-05bf-4d4f-b383-13903e91464d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" Jul 20 05:07:27.456461 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.456351 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/795ef04e-05bf-4d4f-b383-13903e91464d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g\" (UID: \"795ef04e-05bf-4d4f-b383-13903e91464d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" Jul 20 05:07:27.456461 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.456385 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z87b\" (UniqueName: \"kubernetes.io/projected/795ef04e-05bf-4d4f-b383-13903e91464d-kube-api-access-8z87b\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g\" (UID: \"795ef04e-05bf-4d4f-b383-13903e91464d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" Jul 20 05:07:27.456655 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.456636 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/795ef04e-05bf-4d4f-b383-13903e91464d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g\" (UID: \"795ef04e-05bf-4d4f-b383-13903e91464d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" Jul 20 05:07:27.456724 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.456704 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/795ef04e-05bf-4d4f-b383-13903e91464d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g\" (UID: \"795ef04e-05bf-4d4f-b383-13903e91464d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" Jul 20 05:07:27.464349 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.464318 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z87b\" (UniqueName: \"kubernetes.io/projected/795ef04e-05bf-4d4f-b383-13903e91464d-kube-api-access-8z87b\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g\" (UID: \"795ef04e-05bf-4d4f-b383-13903e91464d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" Jul 20 05:07:27.559751 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.559664 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" Jul 20 05:07:27.693067 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:27.693040 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g"] Jul 20 05:07:27.695830 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:07:27.695796 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795ef04e_05bf_4d4f_b383_13903e91464d.slice/crio-36240b8579c6e3ecc37aa0e0a004733fb91101195d2ff1b572238e1426aa1532 WatchSource:0}: Error finding container 36240b8579c6e3ecc37aa0e0a004733fb91101195d2ff1b572238e1426aa1532: Status 404 returned error can't find the container with id 36240b8579c6e3ecc37aa0e0a004733fb91101195d2ff1b572238e1426aa1532 Jul 20 05:07:28.601440 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:28.601411 2576 generic.go:356] "Generic (PLEG): container finished" podID="795ef04e-05bf-4d4f-b383-13903e91464d" containerID="d62582f6a4f9b65339a8c80b612fe2a6b7e4561091971316712d6b6e6228d404" exitCode=0 Jul 20 05:07:28.601819 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:28.601497 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" event={"ID":"795ef04e-05bf-4d4f-b383-13903e91464d","Type":"ContainerDied","Data":"d62582f6a4f9b65339a8c80b612fe2a6b7e4561091971316712d6b6e6228d404"} Jul 20 05:07:28.601819 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:28.601555 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" event={"ID":"795ef04e-05bf-4d4f-b383-13903e91464d","Type":"ContainerStarted","Data":"36240b8579c6e3ecc37aa0e0a004733fb91101195d2ff1b572238e1426aa1532"} Jul 20 05:07:29.607604 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:29.607511 2576 generic.go:356] "Generic (PLEG): container finished" podID="795ef04e-05bf-4d4f-b383-13903e91464d" containerID="7e67527d9165d57af086183f2765df101386021c3d7c06fbbb1d71634d000382" exitCode=0 Jul 20 05:07:29.607604 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:29.607588 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" event={"ID":"795ef04e-05bf-4d4f-b383-13903e91464d","Type":"ContainerDied","Data":"7e67527d9165d57af086183f2765df101386021c3d7c06fbbb1d71634d000382"} Jul 20 05:07:30.614990 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:30.614934 2576 generic.go:356] "Generic (PLEG): container finished" podID="795ef04e-05bf-4d4f-b383-13903e91464d" containerID="2cf91742560e4d914a0a68677d9160f99db4b553c78b7224afe8dd23a213a9a1" exitCode=0 Jul 20 05:07:30.615469 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:30.615025 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" event={"ID":"795ef04e-05bf-4d4f-b383-13903e91464d","Type":"ContainerDied","Data":"2cf91742560e4d914a0a68677d9160f99db4b553c78b7224afe8dd23a213a9a1"} Jul 20 05:07:31.229361 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.229321 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:31.296745 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.296708 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb"] Jul 20 05:07:31.300012 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.299993 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.302555 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.302532 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:31.303061 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.303040 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 20 05:07:31.303139 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.303086 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 20 05:07:31.303333 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.303319 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 20 05:07:31.304185 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.304169 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-b7gl5\"" Jul 20 05:07:31.304257 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.304184 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 20 05:07:31.304257 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.304174 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 20 05:07:31.320942 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.320912 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb"] Jul 20 05:07:31.390618 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.390567 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/989baa4d-2079-4239-a916-7a3187dbf2cd-manager-config\") pod \"lws-controller-manager-676c6f859f-cv6hb\" (UID: \"989baa4d-2079-4239-a916-7a3187dbf2cd\") " pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.390792 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.390652 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2685z\" (UniqueName: \"kubernetes.io/projected/989baa4d-2079-4239-a916-7a3187dbf2cd-kube-api-access-2685z\") pod \"lws-controller-manager-676c6f859f-cv6hb\" (UID: \"989baa4d-2079-4239-a916-7a3187dbf2cd\") " pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.390792 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.390731 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/989baa4d-2079-4239-a916-7a3187dbf2cd-cert\") pod \"lws-controller-manager-676c6f859f-cv6hb\" (UID: \"989baa4d-2079-4239-a916-7a3187dbf2cd\") " pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.390792 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.390756 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/989baa4d-2079-4239-a916-7a3187dbf2cd-metrics-cert\") pod \"lws-controller-manager-676c6f859f-cv6hb\" (UID: \"989baa4d-2079-4239-a916-7a3187dbf2cd\") " pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.491430 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.491396 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2685z\" (UniqueName: \"kubernetes.io/projected/989baa4d-2079-4239-a916-7a3187dbf2cd-kube-api-access-2685z\") pod \"lws-controller-manager-676c6f859f-cv6hb\" (UID: \"989baa4d-2079-4239-a916-7a3187dbf2cd\") " pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.491567 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.491506 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/989baa4d-2079-4239-a916-7a3187dbf2cd-cert\") pod \"lws-controller-manager-676c6f859f-cv6hb\" (UID: \"989baa4d-2079-4239-a916-7a3187dbf2cd\") " pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.491567 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.491535 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/989baa4d-2079-4239-a916-7a3187dbf2cd-metrics-cert\") pod \"lws-controller-manager-676c6f859f-cv6hb\" (UID: \"989baa4d-2079-4239-a916-7a3187dbf2cd\") " pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.491658 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.491596 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/989baa4d-2079-4239-a916-7a3187dbf2cd-manager-config\") pod \"lws-controller-manager-676c6f859f-cv6hb\" (UID: \"989baa4d-2079-4239-a916-7a3187dbf2cd\") " pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.492281 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.492260 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/989baa4d-2079-4239-a916-7a3187dbf2cd-manager-config\") pod \"lws-controller-manager-676c6f859f-cv6hb\" (UID: \"989baa4d-2079-4239-a916-7a3187dbf2cd\") " pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.494001 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.493976 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/989baa4d-2079-4239-a916-7a3187dbf2cd-cert\") pod \"lws-controller-manager-676c6f859f-cv6hb\" (UID: \"989baa4d-2079-4239-a916-7a3187dbf2cd\") " pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.494154 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.494134 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/989baa4d-2079-4239-a916-7a3187dbf2cd-metrics-cert\") pod \"lws-controller-manager-676c6f859f-cv6hb\" (UID: \"989baa4d-2079-4239-a916-7a3187dbf2cd\") " pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.499828 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.499808 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2685z\" (UniqueName: \"kubernetes.io/projected/989baa4d-2079-4239-a916-7a3187dbf2cd-kube-api-access-2685z\") pod \"lws-controller-manager-676c6f859f-cv6hb\" (UID: \"989baa4d-2079-4239-a916-7a3187dbf2cd\") " pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.610095 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.610040 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:31.806027 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.806003 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" Jul 20 05:07:31.813964 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.813920 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb"] Jul 20 05:07:31.817012 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:07:31.816982 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod989baa4d_2079_4239_a916_7a3187dbf2cd.slice/crio-5052034516a74bf3a0bd51246b863f6da3b98affe80b22ad5ab86081050f5166 WatchSource:0}: Error finding container 5052034516a74bf3a0bd51246b863f6da3b98affe80b22ad5ab86081050f5166: Status 404 returned error can't find the container with id 5052034516a74bf3a0bd51246b863f6da3b98affe80b22ad5ab86081050f5166 Jul 20 05:07:31.895833 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.895798 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/795ef04e-05bf-4d4f-b383-13903e91464d-bundle\") pod \"795ef04e-05bf-4d4f-b383-13903e91464d\" (UID: \"795ef04e-05bf-4d4f-b383-13903e91464d\") " Jul 20 05:07:31.896033 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.895844 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/795ef04e-05bf-4d4f-b383-13903e91464d-util\") pod \"795ef04e-05bf-4d4f-b383-13903e91464d\" (UID: \"795ef04e-05bf-4d4f-b383-13903e91464d\") " Jul 20 05:07:31.896033 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.895874 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z87b\" (UniqueName: \"kubernetes.io/projected/795ef04e-05bf-4d4f-b383-13903e91464d-kube-api-access-8z87b\") pod \"795ef04e-05bf-4d4f-b383-13903e91464d\" (UID: \"795ef04e-05bf-4d4f-b383-13903e91464d\") " Jul 20 05:07:31.896564 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.896536 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795ef04e-05bf-4d4f-b383-13903e91464d-bundle" (OuterVolumeSpecName: "bundle") pod "795ef04e-05bf-4d4f-b383-13903e91464d" (UID: "795ef04e-05bf-4d4f-b383-13903e91464d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:07:31.897946 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.897924 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/795ef04e-05bf-4d4f-b383-13903e91464d-kube-api-access-8z87b" (OuterVolumeSpecName: "kube-api-access-8z87b") pod "795ef04e-05bf-4d4f-b383-13903e91464d" (UID: "795ef04e-05bf-4d4f-b383-13903e91464d"). InnerVolumeSpecName "kube-api-access-8z87b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:07:31.901587 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.901552 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795ef04e-05bf-4d4f-b383-13903e91464d-util" (OuterVolumeSpecName: "util") pod "795ef04e-05bf-4d4f-b383-13903e91464d" (UID: "795ef04e-05bf-4d4f-b383-13903e91464d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:07:31.996788 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.996748 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/795ef04e-05bf-4d4f-b383-13903e91464d-bundle\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:07:31.996788 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.996781 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/795ef04e-05bf-4d4f-b383-13903e91464d-util\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:07:31.996788 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:31.996793 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z87b\" (UniqueName: \"kubernetes.io/projected/795ef04e-05bf-4d4f-b383-13903e91464d-kube-api-access-8z87b\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:07:32.627824 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:32.627789 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" event={"ID":"989baa4d-2079-4239-a916-7a3187dbf2cd","Type":"ContainerStarted","Data":"5052034516a74bf3a0bd51246b863f6da3b98affe80b22ad5ab86081050f5166"} Jul 20 05:07:32.629788 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:32.629755 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" event={"ID":"795ef04e-05bf-4d4f-b383-13903e91464d","Type":"ContainerDied","Data":"36240b8579c6e3ecc37aa0e0a004733fb91101195d2ff1b572238e1426aa1532"} Jul 20 05:07:32.629911 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:32.629793 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36240b8579c6e3ecc37aa0e0a004733fb91101195d2ff1b572238e1426aa1532" Jul 20 05:07:32.629911 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:32.629794 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpth6g" Jul 20 05:07:34.642659 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:34.642624 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" event={"ID":"989baa4d-2079-4239-a916-7a3187dbf2cd","Type":"ContainerStarted","Data":"044efcfcadaf51c55fbe124443b7820301208d42e41e314ad8a91b16a1c0e96c"} Jul 20 05:07:34.643104 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:34.642747 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:34.659382 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:34.659323 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" podStartSLOduration=1.767184063 podStartE2EDuration="3.659304447s" totalImagesPullingTime="1.892120384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:07:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:07:34.657775983 +0000 UTC m=+497.704302906" watchObservedRunningTime="2026-07-20 05:07:34.659304447 +0000 UTC m=+497.705831353" Jul 20 05:07:38.775177 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:38.775135 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:38.878717 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:38.878682 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:39.393437 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.393402 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:39.628023 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.627990 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:39.732060 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.732032 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:39.779214 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.779181 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd"] Jul 20 05:07:39.779585 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.779502 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="795ef04e-05bf-4d4f-b383-13903e91464d" containerName="extract" Jul 20 05:07:39.779585 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.779513 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="795ef04e-05bf-4d4f-b383-13903e91464d" containerName="extract" Jul 20 05:07:39.779585 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.779525 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="795ef04e-05bf-4d4f-b383-13903e91464d" containerName="util" Jul 20 05:07:39.779585 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.779530 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="795ef04e-05bf-4d4f-b383-13903e91464d" containerName="util" Jul 20 05:07:39.779585 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.779542 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="795ef04e-05bf-4d4f-b383-13903e91464d" containerName="pull" Jul 20 05:07:39.779585 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.779547 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="795ef04e-05bf-4d4f-b383-13903e91464d" containerName="pull" Jul 20 05:07:39.779778 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.779608 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="795ef04e-05bf-4d4f-b383-13903e91464d" containerName="extract" Jul 20 05:07:39.782777 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.782761 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.785365 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.785342 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5gbtj\"" Jul 20 05:07:39.785365 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.785355 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 20 05:07:39.785687 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.785669 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 20 05:07:39.785808 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.785792 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 20 05:07:39.785881 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.785846 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 20 05:07:39.799159 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.799132 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd"] Jul 20 05:07:39.867519 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.867480 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-manifests\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.867519 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.867517 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-tmp\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.867778 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.867540 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjtq4\" (UniqueName: \"kubernetes.io/projected/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-kube-api-access-pjtq4\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.867778 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.867618 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.867778 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.867729 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.884248 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.884216 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:39.968727 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.968685 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.968917 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.968775 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.968917 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.968799 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-manifests\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.968917 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.968822 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-tmp\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.968917 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.968846 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjtq4\" (UniqueName: \"kubernetes.io/projected/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-kube-api-access-pjtq4\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.969329 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.969301 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-manifests\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.971222 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.971196 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-tmp\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.971489 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.971470 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.971595 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.971572 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.976612 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.976593 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjtq4\" (UniqueName: \"kubernetes.io/projected/0f0bfbb7-fe06-4730-8a34-41368c5d39ce-kube-api-access-pjtq4\") pod \"opendatahub-operator-controller-manager-54dff8fcf7-2c2gd\" (UID: \"0f0bfbb7-fe06-4730-8a34-41368c5d39ce\") " pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:39.984688 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:39.984603 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:40.084491 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:40.084454 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:40.094707 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:40.094683 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:40.191427 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:40.191395 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:40.249092 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:40.249068 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd"] Jul 20 05:07:40.252042 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:07:40.252017 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0bfbb7_fe06_4730_8a34_41368c5d39ce.slice/crio-d0b8148a5499b35cd85082a9e0448d01fdae1cfdbd0f82e2ea0b897df38a5b99 WatchSource:0}: Error finding container d0b8148a5499b35cd85082a9e0448d01fdae1cfdbd0f82e2ea0b897df38a5b99: Status 404 returned error can't find the container with id d0b8148a5499b35cd85082a9e0448d01fdae1cfdbd0f82e2ea0b897df38a5b99 Jul 20 05:07:40.303553 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:40.303523 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:40.388270 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:40.388241 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:40.671069 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:40.670974 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" event={"ID":"0f0bfbb7-fe06-4730-8a34-41368c5d39ce","Type":"ContainerStarted","Data":"d0b8148a5499b35cd85082a9e0448d01fdae1cfdbd0f82e2ea0b897df38a5b99"} Jul 20 05:07:43.689078 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:43.689040 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" event={"ID":"0f0bfbb7-fe06-4730-8a34-41368c5d39ce","Type":"ContainerStarted","Data":"4c569fdd3935a2dec7228a448f3fb3e359345db874289a3ba05e97792ce0a385"} Jul 20 05:07:44.695151 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:44.695066 2576 generic.go:356] "Generic (PLEG): container finished" podID="0f0bfbb7-fe06-4730-8a34-41368c5d39ce" containerID="4c569fdd3935a2dec7228a448f3fb3e359345db874289a3ba05e97792ce0a385" exitCode=0 Jul 20 05:07:44.695500 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:44.695155 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" event={"ID":"0f0bfbb7-fe06-4730-8a34-41368c5d39ce","Type":"ContainerDied","Data":"4c569fdd3935a2dec7228a448f3fb3e359345db874289a3ba05e97792ce0a385"} Jul 20 05:07:44.761804 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:44.761756 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:44.861779 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:44.861732 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:45.652894 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:45.652865 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-676c6f859f-cv6hb" Jul 20 05:07:45.701937 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:45.701904 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" event={"ID":"0f0bfbb7-fe06-4730-8a34-41368c5d39ce","Type":"ContainerStarted","Data":"34817c38a13a2f684e5eb883cf9fd2ff20dc9c87b8645db8612355cd26413156"} Jul 20 05:07:45.702421 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:45.702108 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:45.725727 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:45.725663 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" podStartSLOduration=2.522080446 podStartE2EDuration="6.725647587s" totalImagesPullingTime="3.134783979s" totalInitContainerRuntime="1.068783162s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:07:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:07:45.724171319 +0000 UTC m=+508.770698276" watchObservedRunningTime="2026-07-20 05:07:45.725647587 +0000 UTC m=+508.772174519" Jul 20 05:07:56.709151 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:56.709117 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-54dff8fcf7-2c2gd" Jul 20 05:07:58.261562 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:58.261522 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:58.354736 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:58.354702 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:58.940865 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:58.940820 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:59.044565 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:59.044533 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:59.141715 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:59.141681 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:59.467191 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:59.467160 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:59.582761 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:59.582729 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:07:59.701049 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:07:59.701015 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:08.262834 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:08.262763 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:08.430258 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:08.430226 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:08.497931 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:08.497894 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:08.668120 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:08.668049 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:09.189678 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:09.189595 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:09.335510 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:09.335474 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:09.595367 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:09.595332 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:09.700522 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:09.700461 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:09.798913 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:09.798871 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:10.860193 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:10.860158 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:10.974404 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:10.974370 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:11.911848 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:11.911805 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:16.332230 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.332198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:16.533888 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.533853 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:16.630452 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.630376 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:16.784168 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.784130 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54"] Jul 20 05:08:16.793861 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.793834 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:16.798879 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.798846 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-tnqp9\"" Jul 20 05:08:16.802945 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.802268 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 20 05:08:16.803850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.803820 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54"] Jul 20 05:08:16.848277 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.848248 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:16.908887 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.908797 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:16.908887 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.908836 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:16.908887 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.908856 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:16.909138 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.908941 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rgbz\" (UniqueName: \"kubernetes.io/projected/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-kube-api-access-9rgbz\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:16.909138 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.909010 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:16.909138 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.909028 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:16.909138 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.909077 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:16.909333 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.909201 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:16.909333 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.909249 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:16.972819 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.972776 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9"] Jul 20 05:08:16.976293 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.976275 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:16.988906 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:16.988873 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9"] Jul 20 05:08:17.009978 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.009918 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.009978 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.009974 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.010227 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.010018 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.010227 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.010036 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.010227 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.010052 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.010438 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.010415 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.010500 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.010441 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.010500 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.010481 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.010564 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.010224 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rgbz\" (UniqueName: \"kubernetes.io/projected/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-kube-api-access-9rgbz\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.010638 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.010618 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.010697 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.010657 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.010697 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.010682 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.010974 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.010938 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.011408 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.011387 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.012418 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.012398 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.012860 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.012838 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.017877 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.017849 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rgbz\" (UniqueName: \"kubernetes.io/projected/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-kube-api-access-9rgbz\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.018261 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.018239 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-kmp54\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.106912 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.106877 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:17.111478 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.111161 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.111478 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.111197 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.111478 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.111225 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.111478 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.111309 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.111478 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.111339 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.111478 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.111362 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.111478 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.111406 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.111478 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.111430 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7tvv\" (UniqueName: \"kubernetes.io/projected/eae43db9-d7d1-4eb1-abd0-0da24b91df53-kube-api-access-j7tvv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.111478 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.111460 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.142623 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.142584 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:17.212432 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.212399 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.212432 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.212434 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.212745 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.212457 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.212745 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.212479 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.212745 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.212505 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7tvv\" (UniqueName: \"kubernetes.io/projected/eae43db9-d7d1-4eb1-abd0-0da24b91df53-kube-api-access-j7tvv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.212745 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.212551 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.212745 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.212616 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.212745 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.212639 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.212745 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.212686 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.213138 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.212879 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.213233 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.213208 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.213751 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.213562 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.213751 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.213703 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.213751 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.213720 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.215317 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.215292 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.215626 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.215606 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.220933 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.220902 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eae43db9-d7d1-4eb1-abd0-0da24b91df53-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.221103 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.221077 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7tvv\" (UniqueName: \"kubernetes.io/projected/eae43db9-d7d1-4eb1-abd0-0da24b91df53-kube-api-access-j7tvv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4w7zs9\" (UID: \"eae43db9-d7d1-4eb1-abd0-0da24b91df53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.239406 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.239372 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:17.251923 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.251898 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54"] Jul 20 05:08:17.254072 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:08:17.254042 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7657ff0e_d080_49c3_b10a_a4c9f0addd9d.slice/crio-a56c3331ffdc36da34840b982cf57d95a6c0b05fc6bca3998638cce10983c946 WatchSource:0}: Error finding container a56c3331ffdc36da34840b982cf57d95a6c0b05fc6bca3998638cce10983c946: Status 404 returned error can't find the container with id a56c3331ffdc36da34840b982cf57d95a6c0b05fc6bca3998638cce10983c946 Jul 20 05:08:17.287574 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.287543 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:17.421419 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.421388 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9"] Jul 20 05:08:17.423553 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:08:17.423522 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeae43db9_d7d1_4eb1_abd0_0da24b91df53.slice/crio-dd141c32b6a02fbecc96b881e6222973916b666d1664544685d6c581a88ec895 WatchSource:0}: Error finding container dd141c32b6a02fbecc96b881e6222973916b666d1664544685d6c581a88ec895: Status 404 returned error can't find the container with id dd141c32b6a02fbecc96b881e6222973916b666d1664544685d6c581a88ec895 Jul 20 05:08:17.862295 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.862250 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" event={"ID":"eae43db9-d7d1-4eb1-abd0-0da24b91df53","Type":"ContainerStarted","Data":"dd141c32b6a02fbecc96b881e6222973916b666d1664544685d6c581a88ec895"} Jul 20 05:08:17.863652 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:17.863619 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" event={"ID":"7657ff0e-d080-49c3-b10a-a4c9f0addd9d","Type":"ContainerStarted","Data":"a56c3331ffdc36da34840b982cf57d95a6c0b05fc6bca3998638cce10983c946"} Jul 20 05:08:20.028648 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:20.028599 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:08:20.029020 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:20.028685 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:08:20.029020 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:20.028717 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:08:20.035545 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:20.035506 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:08:20.035652 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:20.035584 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:08:20.035652 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:20.035612 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:08:20.881985 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:20.881919 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" event={"ID":"eae43db9-d7d1-4eb1-abd0-0da24b91df53","Type":"ContainerStarted","Data":"794d1e369327dc83d541e61e1845759fa1e050a45a246a36d42daf050921cf46"} Jul 20 05:08:20.883499 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:20.883467 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" event={"ID":"7657ff0e-d080-49c3-b10a-a4c9f0addd9d","Type":"ContainerStarted","Data":"de347fa98b04611a076e4efdc8764cc3b05887df546bfcb0101cbcdb36974fb0"} Jul 20 05:08:20.901524 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:20.901474 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" podStartSLOduration=2.291810137 podStartE2EDuration="4.901459903s" totalImagesPullingTime="2.609649766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:08:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:08:20.89978403 +0000 UTC m=+543.946310981" watchObservedRunningTime="2026-07-20 05:08:20.901459903 +0000 UTC m=+543.947986809" Jul 20 05:08:20.916767 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:20.916712 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" podStartSLOduration=2.144373661 podStartE2EDuration="4.916698072s" totalImagesPullingTime="2.772324411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:08:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:08:20.916290635 +0000 UTC m=+543.962817568" watchObservedRunningTime="2026-07-20 05:08:20.916698072 +0000 UTC m=+543.963224977" Jul 20 05:08:21.107028 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:21.106995 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:21.108398 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:21.108367 2576 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 20 05:08:21.108507 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:21.108420 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" podUID="7657ff0e-d080-49c3-b10a-a4c9f0addd9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 20 05:08:21.288146 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:21.288109 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:21.343364 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:21.343331 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:21.888437 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:21.888397 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:21.889345 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:21.889327 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4w7zs9" Jul 20 05:08:21.937238 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:21.937198 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54"] Jul 20 05:08:22.107479 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:22.107436 2576 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 20 05:08:22.107842 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:22.107501 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" podUID="7657ff0e-d080-49c3-b10a-a4c9f0addd9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 20 05:08:23.107304 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:23.107265 2576 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 20 05:08:23.107473 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:23.107321 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" podUID="7657ff0e-d080-49c3-b10a-a4c9f0addd9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 20 05:08:23.898087 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:23.898044 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" podUID="7657ff0e-d080-49c3-b10a-a4c9f0addd9d" containerName="istio-proxy" containerID="cri-o://de347fa98b04611a076e4efdc8764cc3b05887df546bfcb0101cbcdb36974fb0" gracePeriod=30 Jul 20 05:08:29.151125 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.151099 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:29.324916 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.324881 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-workload-certs\") pod \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " Jul 20 05:08:29.325116 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.324987 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-envoy\") pod \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " Jul 20 05:08:29.325116 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325020 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-workload-socket\") pod \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " Jul 20 05:08:29.325116 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325042 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-podinfo\") pod \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " Jul 20 05:08:29.325116 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325064 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istiod-ca-cert\") pod \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " Jul 20 05:08:29.325116 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325091 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-data\") pod \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " Jul 20 05:08:29.325378 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325257 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7657ff0e-d080-49c3-b10a-a4c9f0addd9d" (UID: "7657ff0e-d080-49c3-b10a-a4c9f0addd9d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:08:29.325378 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325277 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-credential-socket\") pod \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " Jul 20 05:08:29.325378 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325315 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rgbz\" (UniqueName: \"kubernetes.io/projected/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-kube-api-access-9rgbz\") pod \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " Jul 20 05:08:29.325378 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325360 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7657ff0e-d080-49c3-b10a-a4c9f0addd9d" (UID: "7657ff0e-d080-49c3-b10a-a4c9f0addd9d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:08:29.325557 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325431 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-token\") pod \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\" (UID: \"7657ff0e-d080-49c3-b10a-a4c9f0addd9d\") " Jul 20 05:08:29.325761 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325716 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-workload-socket\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:08:29.325761 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325739 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-workload-certs\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:08:29.325903 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325830 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-data" (OuterVolumeSpecName: "istio-data") pod "7657ff0e-d080-49c3-b10a-a4c9f0addd9d" (UID: "7657ff0e-d080-49c3-b10a-a4c9f0addd9d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:08:29.325979 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325914 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7657ff0e-d080-49c3-b10a-a4c9f0addd9d" (UID: "7657ff0e-d080-49c3-b10a-a4c9f0addd9d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:08:29.325979 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.325921 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7657ff0e-d080-49c3-b10a-a4c9f0addd9d" (UID: "7657ff0e-d080-49c3-b10a-a4c9f0addd9d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:08:29.327832 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.327802 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-token" (OuterVolumeSpecName: "istio-token") pod "7657ff0e-d080-49c3-b10a-a4c9f0addd9d" (UID: "7657ff0e-d080-49c3-b10a-a4c9f0addd9d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:08:29.327928 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.327844 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7657ff0e-d080-49c3-b10a-a4c9f0addd9d" (UID: "7657ff0e-d080-49c3-b10a-a4c9f0addd9d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 05:08:29.327928 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.327860 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-kube-api-access-9rgbz" (OuterVolumeSpecName: "kube-api-access-9rgbz") pod "7657ff0e-d080-49c3-b10a-a4c9f0addd9d" (UID: "7657ff0e-d080-49c3-b10a-a4c9f0addd9d"). InnerVolumeSpecName "kube-api-access-9rgbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:08:29.327928 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.327891 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7657ff0e-d080-49c3-b10a-a4c9f0addd9d" (UID: "7657ff0e-d080-49c3-b10a-a4c9f0addd9d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:08:29.426588 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.426544 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-token\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:08:29.426588 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.426580 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-envoy\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:08:29.426588 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.426589 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-podinfo\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:08:29.426588 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.426600 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istiod-ca-cert\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:08:29.426838 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.426607 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-istio-data\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:08:29.426838 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.426615 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-credential-socket\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:08:29.426838 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.426625 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rgbz\" (UniqueName: \"kubernetes.io/projected/7657ff0e-d080-49c3-b10a-a4c9f0addd9d-kube-api-access-9rgbz\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:08:29.926952 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.926873 2576 generic.go:356] "Generic (PLEG): container finished" podID="7657ff0e-d080-49c3-b10a-a4c9f0addd9d" containerID="de347fa98b04611a076e4efdc8764cc3b05887df546bfcb0101cbcdb36974fb0" exitCode=0 Jul 20 05:08:29.926952 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.926916 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" Jul 20 05:08:29.926952 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.926925 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" event={"ID":"7657ff0e-d080-49c3-b10a-a4c9f0addd9d","Type":"ContainerDied","Data":"de347fa98b04611a076e4efdc8764cc3b05887df546bfcb0101cbcdb36974fb0"} Jul 20 05:08:29.927196 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.926977 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54" event={"ID":"7657ff0e-d080-49c3-b10a-a4c9f0addd9d","Type":"ContainerDied","Data":"a56c3331ffdc36da34840b982cf57d95a6c0b05fc6bca3998638cce10983c946"} Jul 20 05:08:29.927196 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.926999 2576 scope.go:117] "RemoveContainer" containerID="de347fa98b04611a076e4efdc8764cc3b05887df546bfcb0101cbcdb36974fb0" Jul 20 05:08:29.935828 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.935810 2576 scope.go:117] "RemoveContainer" containerID="de347fa98b04611a076e4efdc8764cc3b05887df546bfcb0101cbcdb36974fb0" Jul 20 05:08:29.936098 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:08:29.936077 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de347fa98b04611a076e4efdc8764cc3b05887df546bfcb0101cbcdb36974fb0\": container with ID starting with de347fa98b04611a076e4efdc8764cc3b05887df546bfcb0101cbcdb36974fb0 not found: ID does not exist" containerID="de347fa98b04611a076e4efdc8764cc3b05887df546bfcb0101cbcdb36974fb0" Jul 20 05:08:29.936178 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.936109 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de347fa98b04611a076e4efdc8764cc3b05887df546bfcb0101cbcdb36974fb0"} err="failed to get container status \"de347fa98b04611a076e4efdc8764cc3b05887df546bfcb0101cbcdb36974fb0\": rpc error: code = NotFound desc = could not find container \"de347fa98b04611a076e4efdc8764cc3b05887df546bfcb0101cbcdb36974fb0\": container with ID starting with de347fa98b04611a076e4efdc8764cc3b05887df546bfcb0101cbcdb36974fb0 not found: ID does not exist" Jul 20 05:08:29.944870 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.944848 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54"] Jul 20 05:08:29.948445 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:29.948422 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-kmp54"] Jul 20 05:08:31.653029 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:31.652996 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7657ff0e-d080-49c3-b10a-a4c9f0addd9d" path="/var/lib/kubelet/pods/7657ff0e-d080-49c3-b10a-a4c9f0addd9d/volumes" Jul 20 05:08:51.486197 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:51.486156 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:52.792671 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:52.792636 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:59.661342 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:59.661299 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:59.762607 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:59.762572 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:59.790210 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:59.790172 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dqpm6"] Jul 20 05:08:59.790571 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:59.790557 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7657ff0e-d080-49c3-b10a-a4c9f0addd9d" containerName="istio-proxy" Jul 20 05:08:59.790627 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:59.790575 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7657ff0e-d080-49c3-b10a-a4c9f0addd9d" containerName="istio-proxy" Jul 20 05:08:59.790677 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:59.790643 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7657ff0e-d080-49c3-b10a-a4c9f0addd9d" containerName="istio-proxy" Jul 20 05:08:59.799213 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:59.799184 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dqpm6" Jul 20 05:08:59.801070 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:59.801046 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dqpm6"] Jul 20 05:08:59.801824 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:59.801800 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 20 05:08:59.802766 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:59.802744 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 20 05:08:59.803050 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:59.802799 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-zg8jn\"" Jul 20 05:08:59.832749 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:59.832720 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:08:59.902744 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:08:59.902706 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj6wd\" (UniqueName: \"kubernetes.io/projected/e84fc528-ac27-42b7-87cc-256fa8aea0ae-kube-api-access-wj6wd\") pod \"kuadrant-operator-catalog-dqpm6\" (UID: \"e84fc528-ac27-42b7-87cc-256fa8aea0ae\") " pod="kuadrant-system/kuadrant-operator-catalog-dqpm6" Jul 20 05:09:00.003891 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.003860 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj6wd\" (UniqueName: \"kubernetes.io/projected/e84fc528-ac27-42b7-87cc-256fa8aea0ae-kube-api-access-wj6wd\") pod \"kuadrant-operator-catalog-dqpm6\" (UID: \"e84fc528-ac27-42b7-87cc-256fa8aea0ae\") " pod="kuadrant-system/kuadrant-operator-catalog-dqpm6" Jul 20 05:09:00.011535 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.011504 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj6wd\" (UniqueName: \"kubernetes.io/projected/e84fc528-ac27-42b7-87cc-256fa8aea0ae-kube-api-access-wj6wd\") pod \"kuadrant-operator-catalog-dqpm6\" (UID: \"e84fc528-ac27-42b7-87cc-256fa8aea0ae\") " pod="kuadrant-system/kuadrant-operator-catalog-dqpm6" Jul 20 05:09:00.112101 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.112052 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dqpm6" Jul 20 05:09:00.165871 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.165843 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dqpm6"] Jul 20 05:09:00.248628 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.248602 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dqpm6"] Jul 20 05:09:00.250509 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:09:00.250482 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode84fc528_ac27_42b7_87cc_256fa8aea0ae.slice/crio-2588e45d51a043359bd8b6d68515e5e19bcd9990aab7759b7f0446e6ffac5000 WatchSource:0}: Error finding container 2588e45d51a043359bd8b6d68515e5e19bcd9990aab7759b7f0446e6ffac5000: Status 404 returned error can't find the container with id 2588e45d51a043359bd8b6d68515e5e19bcd9990aab7759b7f0446e6ffac5000 Jul 20 05:09:00.372242 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.372206 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zkzpl"] Jul 20 05:09:00.376857 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.376830 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zkzpl" Jul 20 05:09:00.383258 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.383233 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zkzpl"] Jul 20 05:09:00.509432 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.509330 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8tnv\" (UniqueName: \"kubernetes.io/projected/a4d8fa3a-2526-4081-a7b8-7154a9efb8fa-kube-api-access-s8tnv\") pod \"kuadrant-operator-catalog-zkzpl\" (UID: \"a4d8fa3a-2526-4081-a7b8-7154a9efb8fa\") " pod="kuadrant-system/kuadrant-operator-catalog-zkzpl" Jul 20 05:09:00.593451 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.593418 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:00.610973 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.610927 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8tnv\" (UniqueName: \"kubernetes.io/projected/a4d8fa3a-2526-4081-a7b8-7154a9efb8fa-kube-api-access-s8tnv\") pod \"kuadrant-operator-catalog-zkzpl\" (UID: \"a4d8fa3a-2526-4081-a7b8-7154a9efb8fa\") " pod="kuadrant-system/kuadrant-operator-catalog-zkzpl" Jul 20 05:09:00.618632 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.618603 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8tnv\" (UniqueName: \"kubernetes.io/projected/a4d8fa3a-2526-4081-a7b8-7154a9efb8fa-kube-api-access-s8tnv\") pod \"kuadrant-operator-catalog-zkzpl\" (UID: \"a4d8fa3a-2526-4081-a7b8-7154a9efb8fa\") " pod="kuadrant-system/kuadrant-operator-catalog-zkzpl" Jul 20 05:09:00.687726 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.687676 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zkzpl" Jul 20 05:09:00.794809 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.794764 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:00.833012 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:00.832927 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zkzpl"] Jul 20 05:09:00.835206 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:09:00.835174 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d8fa3a_2526_4081_a7b8_7154a9efb8fa.slice/crio-be8d23857711908fb0055561461afd699de9eadf346d42cdea3120af037cd907 WatchSource:0}: Error finding container be8d23857711908fb0055561461afd699de9eadf346d42cdea3120af037cd907: Status 404 returned error can't find the container with id be8d23857711908fb0055561461afd699de9eadf346d42cdea3120af037cd907 Jul 20 05:09:01.076636 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:01.076553 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dqpm6" event={"ID":"e84fc528-ac27-42b7-87cc-256fa8aea0ae","Type":"ContainerStarted","Data":"2588e45d51a043359bd8b6d68515e5e19bcd9990aab7759b7f0446e6ffac5000"} Jul 20 05:09:01.077681 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:01.077658 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zkzpl" event={"ID":"a4d8fa3a-2526-4081-a7b8-7154a9efb8fa","Type":"ContainerStarted","Data":"be8d23857711908fb0055561461afd699de9eadf346d42cdea3120af037cd907"} Jul 20 05:09:03.089907 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:03.089866 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dqpm6" event={"ID":"e84fc528-ac27-42b7-87cc-256fa8aea0ae","Type":"ContainerStarted","Data":"27cde96852f5c8f4d7bb1622684c156ddbb04e5ec10433b1b1a0771b4416481c"} Jul 20 05:09:03.090469 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:03.089919 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-dqpm6" podUID="e84fc528-ac27-42b7-87cc-256fa8aea0ae" containerName="registry-server" containerID="cri-o://27cde96852f5c8f4d7bb1622684c156ddbb04e5ec10433b1b1a0771b4416481c" gracePeriod=2 Jul 20 05:09:03.091355 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:03.091330 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zkzpl" event={"ID":"a4d8fa3a-2526-4081-a7b8-7154a9efb8fa","Type":"ContainerStarted","Data":"3c1b88cb6443745208f70c54b699f1447e80668c79d89506c9a164d073b9ee25"} Jul 20 05:09:03.107482 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:03.107434 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dqpm6" podStartSLOduration=1.781275694 podStartE2EDuration="4.107416933s" totalImagesPullingTime="2.326141239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:09:03.106139865 +0000 UTC m=+586.152666832" watchObservedRunningTime="2026-07-20 05:09:03.107416933 +0000 UTC m=+586.153943838" Jul 20 05:09:03.121726 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:03.121676 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-zkzpl" podStartSLOduration=1.379403804 podStartE2EDuration="3.121659019s" totalImagesPullingTime="1.742255215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:09:03.119689111 +0000 UTC m=+586.166216043" watchObservedRunningTime="2026-07-20 05:09:03.121659019 +0000 UTC m=+586.168185926" Jul 20 05:09:03.342016 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:03.341923 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dqpm6" Jul 20 05:09:03.438722 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:03.438687 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj6wd\" (UniqueName: \"kubernetes.io/projected/e84fc528-ac27-42b7-87cc-256fa8aea0ae-kube-api-access-wj6wd\") pod \"e84fc528-ac27-42b7-87cc-256fa8aea0ae\" (UID: \"e84fc528-ac27-42b7-87cc-256fa8aea0ae\") " Jul 20 05:09:03.440886 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:03.440852 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e84fc528-ac27-42b7-87cc-256fa8aea0ae-kube-api-access-wj6wd" (OuterVolumeSpecName: "kube-api-access-wj6wd") pod "e84fc528-ac27-42b7-87cc-256fa8aea0ae" (UID: "e84fc528-ac27-42b7-87cc-256fa8aea0ae"). InnerVolumeSpecName "kube-api-access-wj6wd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:09:03.539979 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:03.539927 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj6wd\" (UniqueName: \"kubernetes.io/projected/e84fc528-ac27-42b7-87cc-256fa8aea0ae-kube-api-access-wj6wd\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:09:04.097884 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:04.097850 2576 generic.go:356] "Generic (PLEG): container finished" podID="e84fc528-ac27-42b7-87cc-256fa8aea0ae" containerID="27cde96852f5c8f4d7bb1622684c156ddbb04e5ec10433b1b1a0771b4416481c" exitCode=0 Jul 20 05:09:04.098378 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:04.097914 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dqpm6" event={"ID":"e84fc528-ac27-42b7-87cc-256fa8aea0ae","Type":"ContainerDied","Data":"27cde96852f5c8f4d7bb1622684c156ddbb04e5ec10433b1b1a0771b4416481c"} Jul 20 05:09:04.098378 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:04.097944 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dqpm6" Jul 20 05:09:04.098378 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:04.097989 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dqpm6" event={"ID":"e84fc528-ac27-42b7-87cc-256fa8aea0ae","Type":"ContainerDied","Data":"2588e45d51a043359bd8b6d68515e5e19bcd9990aab7759b7f0446e6ffac5000"} Jul 20 05:09:04.098378 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:04.098019 2576 scope.go:117] "RemoveContainer" containerID="27cde96852f5c8f4d7bb1622684c156ddbb04e5ec10433b1b1a0771b4416481c" Jul 20 05:09:04.107379 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:04.107358 2576 scope.go:117] "RemoveContainer" containerID="27cde96852f5c8f4d7bb1622684c156ddbb04e5ec10433b1b1a0771b4416481c" Jul 20 05:09:04.107664 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:09:04.107642 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27cde96852f5c8f4d7bb1622684c156ddbb04e5ec10433b1b1a0771b4416481c\": container with ID starting with 27cde96852f5c8f4d7bb1622684c156ddbb04e5ec10433b1b1a0771b4416481c not found: ID does not exist" containerID="27cde96852f5c8f4d7bb1622684c156ddbb04e5ec10433b1b1a0771b4416481c" Jul 20 05:09:04.107709 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:04.107673 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27cde96852f5c8f4d7bb1622684c156ddbb04e5ec10433b1b1a0771b4416481c"} err="failed to get container status \"27cde96852f5c8f4d7bb1622684c156ddbb04e5ec10433b1b1a0771b4416481c\": rpc error: code = NotFound desc = could not find container \"27cde96852f5c8f4d7bb1622684c156ddbb04e5ec10433b1b1a0771b4416481c\": container with ID starting with 27cde96852f5c8f4d7bb1622684c156ddbb04e5ec10433b1b1a0771b4416481c not found: ID does not exist" Jul 20 05:09:04.113422 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:04.113396 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dqpm6"] Jul 20 05:09:04.117406 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:04.117383 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dqpm6"] Jul 20 05:09:05.652933 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:05.652898 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e84fc528-ac27-42b7-87cc-256fa8aea0ae" path="/var/lib/kubelet/pods/e84fc528-ac27-42b7-87cc-256fa8aea0ae/volumes" Jul 20 05:09:10.688379 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:10.688339 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-zkzpl" Jul 20 05:09:10.688818 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:10.688406 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-zkzpl" Jul 20 05:09:10.711708 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:10.711675 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-zkzpl" Jul 20 05:09:11.155352 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:11.155323 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-zkzpl" Jul 20 05:09:15.005810 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.005770 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74"] Jul 20 05:09:15.010879 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.006788 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e84fc528-ac27-42b7-87cc-256fa8aea0ae" containerName="registry-server" Jul 20 05:09:15.010879 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.006825 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e84fc528-ac27-42b7-87cc-256fa8aea0ae" containerName="registry-server" Jul 20 05:09:15.010879 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.007082 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e84fc528-ac27-42b7-87cc-256fa8aea0ae" containerName="registry-server" Jul 20 05:09:15.013145 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.013119 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" Jul 20 05:09:15.016597 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.016572 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wx9pn\"" Jul 20 05:09:15.018225 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.018201 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74"] Jul 20 05:09:15.144776 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.144737 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a4ced7e-c8df-47fb-86c6-b65d31593b06-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74\" (UID: \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" Jul 20 05:09:15.145008 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.144784 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a4ced7e-c8df-47fb-86c6-b65d31593b06-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74\" (UID: \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" Jul 20 05:09:15.145008 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.144812 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnrzn\" (UniqueName: \"kubernetes.io/projected/3a4ced7e-c8df-47fb-86c6-b65d31593b06-kube-api-access-gnrzn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74\" (UID: \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" Jul 20 05:09:15.245921 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.245875 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a4ced7e-c8df-47fb-86c6-b65d31593b06-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74\" (UID: \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" Jul 20 05:09:15.245921 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.245919 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a4ced7e-c8df-47fb-86c6-b65d31593b06-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74\" (UID: \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" Jul 20 05:09:15.246167 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.245939 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnrzn\" (UniqueName: \"kubernetes.io/projected/3a4ced7e-c8df-47fb-86c6-b65d31593b06-kube-api-access-gnrzn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74\" (UID: \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" Jul 20 05:09:15.246411 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.246384 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a4ced7e-c8df-47fb-86c6-b65d31593b06-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74\" (UID: \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" Jul 20 05:09:15.246447 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.246391 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a4ced7e-c8df-47fb-86c6-b65d31593b06-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74\" (UID: \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" Jul 20 05:09:15.254393 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.254354 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnrzn\" (UniqueName: \"kubernetes.io/projected/3a4ced7e-c8df-47fb-86c6-b65d31593b06-kube-api-access-gnrzn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74\" (UID: \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" Jul 20 05:09:15.324523 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.324430 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" Jul 20 05:09:15.463447 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.463417 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74"] Jul 20 05:09:15.465868 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:09:15.465841 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a4ced7e_c8df_47fb_86c6_b65d31593b06.slice/crio-6b7e7a59d69bdf41156f995649da1987d4289e22d740f73f05794ed95c68cbfc WatchSource:0}: Error finding container 6b7e7a59d69bdf41156f995649da1987d4289e22d740f73f05794ed95c68cbfc: Status 404 returned error can't find the container with id 6b7e7a59d69bdf41156f995649da1987d4289e22d740f73f05794ed95c68cbfc Jul 20 05:09:15.600998 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.600946 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph"] Jul 20 05:09:15.604505 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.604486 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" Jul 20 05:09:15.611337 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.611310 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph"] Jul 20 05:09:15.750730 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.750692 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e245d100-85e9-47d3-bda0-e7f8367b2ad7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph\" (UID: \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" Jul 20 05:09:15.750730 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.750731 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e245d100-85e9-47d3-bda0-e7f8367b2ad7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph\" (UID: \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" Jul 20 05:09:15.751049 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.750815 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dplcr\" (UniqueName: \"kubernetes.io/projected/e245d100-85e9-47d3-bda0-e7f8367b2ad7-kube-api-access-dplcr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph\" (UID: \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" Jul 20 05:09:15.851732 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.851629 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e245d100-85e9-47d3-bda0-e7f8367b2ad7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph\" (UID: \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" Jul 20 05:09:15.851732 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.851675 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e245d100-85e9-47d3-bda0-e7f8367b2ad7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph\" (UID: \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" Jul 20 05:09:15.851732 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.851703 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dplcr\" (UniqueName: \"kubernetes.io/projected/e245d100-85e9-47d3-bda0-e7f8367b2ad7-kube-api-access-dplcr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph\" (UID: \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" Jul 20 05:09:15.852040 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.852020 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e245d100-85e9-47d3-bda0-e7f8367b2ad7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph\" (UID: \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" Jul 20 05:09:15.852085 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.852051 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e245d100-85e9-47d3-bda0-e7f8367b2ad7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph\" (UID: \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" Jul 20 05:09:15.859635 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.859613 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dplcr\" (UniqueName: \"kubernetes.io/projected/e245d100-85e9-47d3-bda0-e7f8367b2ad7-kube-api-access-dplcr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph\" (UID: \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" Jul 20 05:09:15.952540 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:15.952495 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" Jul 20 05:09:16.087111 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.087078 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph"] Jul 20 05:09:16.089227 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:09:16.089199 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode245d100_85e9_47d3_bda0_e7f8367b2ad7.slice/crio-c98c6c9804b2f29b7bbdc7b0ae0769837049ada0d81c686f2934663161ea1f6e WatchSource:0}: Error finding container c98c6c9804b2f29b7bbdc7b0ae0769837049ada0d81c686f2934663161ea1f6e: Status 404 returned error can't find the container with id c98c6c9804b2f29b7bbdc7b0ae0769837049ada0d81c686f2934663161ea1f6e Jul 20 05:09:16.158784 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.158750 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" event={"ID":"e245d100-85e9-47d3-bda0-e7f8367b2ad7","Type":"ContainerStarted","Data":"c98c6c9804b2f29b7bbdc7b0ae0769837049ada0d81c686f2934663161ea1f6e"} Jul 20 05:09:16.160244 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.160217 2576 generic.go:356] "Generic (PLEG): container finished" podID="3a4ced7e-c8df-47fb-86c6-b65d31593b06" containerID="0927030b3710875818928f0ab9bf64fc748bf3bddf3ebebaaf40633acc0b40ab" exitCode=0 Jul 20 05:09:16.160368 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.160266 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" event={"ID":"3a4ced7e-c8df-47fb-86c6-b65d31593b06","Type":"ContainerDied","Data":"0927030b3710875818928f0ab9bf64fc748bf3bddf3ebebaaf40633acc0b40ab"} Jul 20 05:09:16.160368 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.160310 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" event={"ID":"3a4ced7e-c8df-47fb-86c6-b65d31593b06","Type":"ContainerStarted","Data":"6b7e7a59d69bdf41156f995649da1987d4289e22d740f73f05794ed95c68cbfc"} Jul 20 05:09:16.204616 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.204581 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn"] Jul 20 05:09:16.208073 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.208056 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" Jul 20 05:09:16.216000 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.215932 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn"] Jul 20 05:09:16.255152 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.255119 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9ccf\" (UniqueName: \"kubernetes.io/projected/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-kube-api-access-b9ccf\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn\" (UID: \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" Jul 20 05:09:16.255311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.255174 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn\" (UID: \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" Jul 20 05:09:16.255311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.255268 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn\" (UID: \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" Jul 20 05:09:16.356337 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.356245 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn\" (UID: \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" Jul 20 05:09:16.356337 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.356328 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9ccf\" (UniqueName: \"kubernetes.io/projected/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-kube-api-access-b9ccf\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn\" (UID: \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" Jul 20 05:09:16.356626 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.356365 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn\" (UID: \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" Jul 20 05:09:16.356729 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.356707 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn\" (UID: \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" Jul 20 05:09:16.356766 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.356734 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn\" (UID: \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" Jul 20 05:09:16.365651 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.365619 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9ccf\" (UniqueName: \"kubernetes.io/projected/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-kube-api-access-b9ccf\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn\" (UID: \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" Jul 20 05:09:16.555138 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.555097 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" Jul 20 05:09:16.609422 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.609386 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78"] Jul 20 05:09:16.615511 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.615423 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" Jul 20 05:09:16.619978 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.619902 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78"] Jul 20 05:09:16.658937 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.658709 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e040e05c-b612-48e9-a442-53b1505c7187-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78\" (UID: \"e040e05c-b612-48e9-a442-53b1505c7187\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" Jul 20 05:09:16.658937 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.658803 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e040e05c-b612-48e9-a442-53b1505c7187-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78\" (UID: \"e040e05c-b612-48e9-a442-53b1505c7187\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" Jul 20 05:09:16.658937 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.658920 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8kdm\" (UniqueName: \"kubernetes.io/projected/e040e05c-b612-48e9-a442-53b1505c7187-kube-api-access-k8kdm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78\" (UID: \"e040e05c-b612-48e9-a442-53b1505c7187\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" Jul 20 05:09:16.696184 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.696149 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn"] Jul 20 05:09:16.698438 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:09:16.698409 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47df011b_30b9_475e_8bdb_f28c2a1cfe8e.slice/crio-156f6b7c1bc4b17be013622132c5bcf4f465d1554a5513e7ce03c96bee85fc80 WatchSource:0}: Error finding container 156f6b7c1bc4b17be013622132c5bcf4f465d1554a5513e7ce03c96bee85fc80: Status 404 returned error can't find the container with id 156f6b7c1bc4b17be013622132c5bcf4f465d1554a5513e7ce03c96bee85fc80 Jul 20 05:09:16.759623 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.759593 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e040e05c-b612-48e9-a442-53b1505c7187-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78\" (UID: \"e040e05c-b612-48e9-a442-53b1505c7187\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" Jul 20 05:09:16.759790 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.759665 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8kdm\" (UniqueName: \"kubernetes.io/projected/e040e05c-b612-48e9-a442-53b1505c7187-kube-api-access-k8kdm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78\" (UID: \"e040e05c-b612-48e9-a442-53b1505c7187\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" Jul 20 05:09:16.759790 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.759721 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e040e05c-b612-48e9-a442-53b1505c7187-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78\" (UID: \"e040e05c-b612-48e9-a442-53b1505c7187\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" Jul 20 05:09:16.759973 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.759936 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e040e05c-b612-48e9-a442-53b1505c7187-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78\" (UID: \"e040e05c-b612-48e9-a442-53b1505c7187\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" Jul 20 05:09:16.760090 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.760072 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e040e05c-b612-48e9-a442-53b1505c7187-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78\" (UID: \"e040e05c-b612-48e9-a442-53b1505c7187\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" Jul 20 05:09:16.769376 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.769347 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8kdm\" (UniqueName: \"kubernetes.io/projected/e040e05c-b612-48e9-a442-53b1505c7187-kube-api-access-k8kdm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78\" (UID: \"e040e05c-b612-48e9-a442-53b1505c7187\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" Jul 20 05:09:16.933015 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:16.932905 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" Jul 20 05:09:17.087704 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.087675 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78"] Jul 20 05:09:17.093870 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:09:17.093838 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode040e05c_b612_48e9_a442_53b1505c7187.slice/crio-c440fa8129ba740ad375ea957868a4712ee7fc40bb08aa77b311626fa9cac3b4 WatchSource:0}: Error finding container c440fa8129ba740ad375ea957868a4712ee7fc40bb08aa77b311626fa9cac3b4: Status 404 returned error can't find the container with id c440fa8129ba740ad375ea957868a4712ee7fc40bb08aa77b311626fa9cac3b4 Jul 20 05:09:17.173947 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.173910 2576 generic.go:356] "Generic (PLEG): container finished" podID="e245d100-85e9-47d3-bda0-e7f8367b2ad7" containerID="3a746c38006902f375683af2004bbabd2ce2d5690356ff66f8d947540dad10eb" exitCode=0 Jul 20 05:09:17.174147 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.173993 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" event={"ID":"e245d100-85e9-47d3-bda0-e7f8367b2ad7","Type":"ContainerDied","Data":"3a746c38006902f375683af2004bbabd2ce2d5690356ff66f8d947540dad10eb"} Jul 20 05:09:17.176044 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.176014 2576 generic.go:356] "Generic (PLEG): container finished" podID="3a4ced7e-c8df-47fb-86c6-b65d31593b06" containerID="ee06bf8dac288295fa79041ca0d44ce4dfe957924bcac35473d4d9f58582272f" exitCode=0 Jul 20 05:09:17.176149 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.176096 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" event={"ID":"3a4ced7e-c8df-47fb-86c6-b65d31593b06","Type":"ContainerDied","Data":"ee06bf8dac288295fa79041ca0d44ce4dfe957924bcac35473d4d9f58582272f"} Jul 20 05:09:17.178101 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.178074 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" event={"ID":"e040e05c-b612-48e9-a442-53b1505c7187","Type":"ContainerStarted","Data":"6de4bc59c66c810ed2c0cfaadb2854d152f320210849523da18737ac12eeb063"} Jul 20 05:09:17.178214 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.178128 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" event={"ID":"e040e05c-b612-48e9-a442-53b1505c7187","Type":"ContainerStarted","Data":"c440fa8129ba740ad375ea957868a4712ee7fc40bb08aa77b311626fa9cac3b4"} Jul 20 05:09:17.179767 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.179746 2576 generic.go:356] "Generic (PLEG): container finished" podID="47df011b-30b9-475e-8bdb-f28c2a1cfe8e" containerID="0cd6748d16ec5c2c09e757024b356bba97629213511ebd07c9640de3ca348071" exitCode=0 Jul 20 05:09:17.179854 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.179825 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" event={"ID":"47df011b-30b9-475e-8bdb-f28c2a1cfe8e","Type":"ContainerDied","Data":"0cd6748d16ec5c2c09e757024b356bba97629213511ebd07c9640de3ca348071"} Jul 20 05:09:17.179933 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.179865 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" event={"ID":"47df011b-30b9-475e-8bdb-f28c2a1cfe8e","Type":"ContainerStarted","Data":"156f6b7c1bc4b17be013622132c5bcf4f465d1554a5513e7ce03c96bee85fc80"} Jul 20 05:09:17.666562 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.666531 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:09:17.668634 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.668606 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:09:17.671310 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.671287 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:09:17.673306 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:17.673284 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:09:18.187121 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:18.187085 2576 generic.go:356] "Generic (PLEG): container finished" podID="e245d100-85e9-47d3-bda0-e7f8367b2ad7" containerID="2a2fd3a38a7c1f0da4f051a85e316aff509f6e420d326ee585448d0b2021f98c" exitCode=0 Jul 20 05:09:18.187592 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:18.187165 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" event={"ID":"e245d100-85e9-47d3-bda0-e7f8367b2ad7","Type":"ContainerDied","Data":"2a2fd3a38a7c1f0da4f051a85e316aff509f6e420d326ee585448d0b2021f98c"} Jul 20 05:09:18.189344 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:18.189321 2576 generic.go:356] "Generic (PLEG): container finished" podID="3a4ced7e-c8df-47fb-86c6-b65d31593b06" containerID="7a2079f7aa0f7a6d0b71f9e2651d0b0447835888b6682a64a0529089fd249ad8" exitCode=0 Jul 20 05:09:18.189454 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:18.189402 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" event={"ID":"3a4ced7e-c8df-47fb-86c6-b65d31593b06","Type":"ContainerDied","Data":"7a2079f7aa0f7a6d0b71f9e2651d0b0447835888b6682a64a0529089fd249ad8"} Jul 20 05:09:18.190852 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:18.190830 2576 generic.go:356] "Generic (PLEG): container finished" podID="e040e05c-b612-48e9-a442-53b1505c7187" containerID="6de4bc59c66c810ed2c0cfaadb2854d152f320210849523da18737ac12eeb063" exitCode=0 Jul 20 05:09:18.190939 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:18.190858 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" event={"ID":"e040e05c-b612-48e9-a442-53b1505c7187","Type":"ContainerDied","Data":"6de4bc59c66c810ed2c0cfaadb2854d152f320210849523da18737ac12eeb063"} Jul 20 05:09:18.192785 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:18.192763 2576 generic.go:356] "Generic (PLEG): container finished" podID="47df011b-30b9-475e-8bdb-f28c2a1cfe8e" containerID="21055643c6ff06195646756754d3ac8144a95ddbc6570cecd6ca54eef38eae74" exitCode=0 Jul 20 05:09:18.192892 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:18.192826 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" event={"ID":"47df011b-30b9-475e-8bdb-f28c2a1cfe8e","Type":"ContainerDied","Data":"21055643c6ff06195646756754d3ac8144a95ddbc6570cecd6ca54eef38eae74"} Jul 20 05:09:19.200292 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.200201 2576 generic.go:356] "Generic (PLEG): container finished" podID="47df011b-30b9-475e-8bdb-f28c2a1cfe8e" containerID="d13a055b465c903b47a747c97b31cd68385a0178898d6111e8514358100d9eb5" exitCode=0 Jul 20 05:09:19.200666 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.200291 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" event={"ID":"47df011b-30b9-475e-8bdb-f28c2a1cfe8e","Type":"ContainerDied","Data":"d13a055b465c903b47a747c97b31cd68385a0178898d6111e8514358100d9eb5"} Jul 20 05:09:19.202350 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.202326 2576 generic.go:356] "Generic (PLEG): container finished" podID="e245d100-85e9-47d3-bda0-e7f8367b2ad7" containerID="94b2eb6cc4f09bdd8c8fd301cf1f2487472a4bb41dcc77e17c1e6742a658a30d" exitCode=0 Jul 20 05:09:19.202454 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.202392 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" event={"ID":"e245d100-85e9-47d3-bda0-e7f8367b2ad7","Type":"ContainerDied","Data":"94b2eb6cc4f09bdd8c8fd301cf1f2487472a4bb41dcc77e17c1e6742a658a30d"} Jul 20 05:09:19.204129 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.204105 2576 generic.go:356] "Generic (PLEG): container finished" podID="e040e05c-b612-48e9-a442-53b1505c7187" containerID="93dc21c432a9569dd71b8742c5974fd75a00d36cfed04227240f8305bd9decf5" exitCode=0 Jul 20 05:09:19.204239 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.204191 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" event={"ID":"e040e05c-b612-48e9-a442-53b1505c7187","Type":"ContainerDied","Data":"93dc21c432a9569dd71b8742c5974fd75a00d36cfed04227240f8305bd9decf5"} Jul 20 05:09:19.347819 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.347792 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" Jul 20 05:09:19.382861 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.382826 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnrzn\" (UniqueName: \"kubernetes.io/projected/3a4ced7e-c8df-47fb-86c6-b65d31593b06-kube-api-access-gnrzn\") pod \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\" (UID: \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\") " Jul 20 05:09:19.383043 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.382927 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a4ced7e-c8df-47fb-86c6-b65d31593b06-bundle\") pod \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\" (UID: \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\") " Jul 20 05:09:19.383043 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.382951 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a4ced7e-c8df-47fb-86c6-b65d31593b06-util\") pod \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\" (UID: \"3a4ced7e-c8df-47fb-86c6-b65d31593b06\") " Jul 20 05:09:19.383590 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.383554 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a4ced7e-c8df-47fb-86c6-b65d31593b06-bundle" (OuterVolumeSpecName: "bundle") pod "3a4ced7e-c8df-47fb-86c6-b65d31593b06" (UID: "3a4ced7e-c8df-47fb-86c6-b65d31593b06"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:09:19.385301 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.385269 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a4ced7e-c8df-47fb-86c6-b65d31593b06-kube-api-access-gnrzn" (OuterVolumeSpecName: "kube-api-access-gnrzn") pod "3a4ced7e-c8df-47fb-86c6-b65d31593b06" (UID: "3a4ced7e-c8df-47fb-86c6-b65d31593b06"). InnerVolumeSpecName "kube-api-access-gnrzn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:09:19.388525 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.388497 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a4ced7e-c8df-47fb-86c6-b65d31593b06-util" (OuterVolumeSpecName: "util") pod "3a4ced7e-c8df-47fb-86c6-b65d31593b06" (UID: "3a4ced7e-c8df-47fb-86c6-b65d31593b06"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:09:19.484171 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.484141 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a4ced7e-c8df-47fb-86c6-b65d31593b06-bundle\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:09:19.484171 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.484170 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a4ced7e-c8df-47fb-86c6-b65d31593b06-util\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:09:19.484365 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:19.484179 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnrzn\" (UniqueName: \"kubernetes.io/projected/3a4ced7e-c8df-47fb-86c6-b65d31593b06-kube-api-access-gnrzn\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:09:20.211409 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.211370 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" event={"ID":"3a4ced7e-c8df-47fb-86c6-b65d31593b06","Type":"ContainerDied","Data":"6b7e7a59d69bdf41156f995649da1987d4289e22d740f73f05794ed95c68cbfc"} Jul 20 05:09:20.211893 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.211429 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b7e7a59d69bdf41156f995649da1987d4289e22d740f73f05794ed95c68cbfc" Jul 20 05:09:20.211893 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.211384 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74" Jul 20 05:09:20.213707 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.213676 2576 generic.go:356] "Generic (PLEG): container finished" podID="e040e05c-b612-48e9-a442-53b1505c7187" containerID="dacf1089cbc8e22a1b82e364e4e2912172d1afc398c1e954304605a81fedbe80" exitCode=0 Jul 20 05:09:20.213837 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.213797 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" event={"ID":"e040e05c-b612-48e9-a442-53b1505c7187","Type":"ContainerDied","Data":"dacf1089cbc8e22a1b82e364e4e2912172d1afc398c1e954304605a81fedbe80"} Jul 20 05:09:20.371360 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.371331 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" Jul 20 05:09:20.373432 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.373411 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" Jul 20 05:09:20.392531 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.392497 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-util\") pod \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\" (UID: \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\") " Jul 20 05:09:20.392678 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.392560 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dplcr\" (UniqueName: \"kubernetes.io/projected/e245d100-85e9-47d3-bda0-e7f8367b2ad7-kube-api-access-dplcr\") pod \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\" (UID: \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\") " Jul 20 05:09:20.392678 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.392619 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9ccf\" (UniqueName: \"kubernetes.io/projected/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-kube-api-access-b9ccf\") pod \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\" (UID: \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\") " Jul 20 05:09:20.392678 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.392659 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e245d100-85e9-47d3-bda0-e7f8367b2ad7-util\") pod \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\" (UID: \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\") " Jul 20 05:09:20.392807 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.392710 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e245d100-85e9-47d3-bda0-e7f8367b2ad7-bundle\") pod \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\" (UID: \"e245d100-85e9-47d3-bda0-e7f8367b2ad7\") " Jul 20 05:09:20.392807 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.392797 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-bundle\") pod \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\" (UID: \"47df011b-30b9-475e-8bdb-f28c2a1cfe8e\") " Jul 20 05:09:20.393344 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.393302 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e245d100-85e9-47d3-bda0-e7f8367b2ad7-bundle" (OuterVolumeSpecName: "bundle") pod "e245d100-85e9-47d3-bda0-e7f8367b2ad7" (UID: "e245d100-85e9-47d3-bda0-e7f8367b2ad7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:09:20.393638 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.393609 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-bundle" (OuterVolumeSpecName: "bundle") pod "47df011b-30b9-475e-8bdb-f28c2a1cfe8e" (UID: "47df011b-30b9-475e-8bdb-f28c2a1cfe8e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:09:20.394932 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.394905 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-kube-api-access-b9ccf" (OuterVolumeSpecName: "kube-api-access-b9ccf") pod "47df011b-30b9-475e-8bdb-f28c2a1cfe8e" (UID: "47df011b-30b9-475e-8bdb-f28c2a1cfe8e"). InnerVolumeSpecName "kube-api-access-b9ccf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:09:20.395170 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.395146 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e245d100-85e9-47d3-bda0-e7f8367b2ad7-kube-api-access-dplcr" (OuterVolumeSpecName: "kube-api-access-dplcr") pod "e245d100-85e9-47d3-bda0-e7f8367b2ad7" (UID: "e245d100-85e9-47d3-bda0-e7f8367b2ad7"). InnerVolumeSpecName "kube-api-access-dplcr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:09:20.399431 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.399402 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-util" (OuterVolumeSpecName: "util") pod "47df011b-30b9-475e-8bdb-f28c2a1cfe8e" (UID: "47df011b-30b9-475e-8bdb-f28c2a1cfe8e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:09:20.401159 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.401126 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e245d100-85e9-47d3-bda0-e7f8367b2ad7-util" (OuterVolumeSpecName: "util") pod "e245d100-85e9-47d3-bda0-e7f8367b2ad7" (UID: "e245d100-85e9-47d3-bda0-e7f8367b2ad7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:09:20.494226 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.494188 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9ccf\" (UniqueName: \"kubernetes.io/projected/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-kube-api-access-b9ccf\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:09:20.494226 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.494223 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e245d100-85e9-47d3-bda0-e7f8367b2ad7-util\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:09:20.494226 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.494233 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e245d100-85e9-47d3-bda0-e7f8367b2ad7-bundle\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:09:20.494449 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.494242 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-bundle\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:09:20.494449 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.494256 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47df011b-30b9-475e-8bdb-f28c2a1cfe8e-util\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:09:20.494449 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:20.494264 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dplcr\" (UniqueName: \"kubernetes.io/projected/e245d100-85e9-47d3-bda0-e7f8367b2ad7-kube-api-access-dplcr\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:09:21.225772 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.225744 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" Jul 20 05:09:21.226219 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.225748 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph" event={"ID":"e245d100-85e9-47d3-bda0-e7f8367b2ad7","Type":"ContainerDied","Data":"c98c6c9804b2f29b7bbdc7b0ae0769837049ada0d81c686f2934663161ea1f6e"} Jul 20 05:09:21.226219 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.225902 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c98c6c9804b2f29b7bbdc7b0ae0769837049ada0d81c686f2934663161ea1f6e" Jul 20 05:09:21.227786 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.227761 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" Jul 20 05:09:21.227786 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.227779 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn" event={"ID":"47df011b-30b9-475e-8bdb-f28c2a1cfe8e","Type":"ContainerDied","Data":"156f6b7c1bc4b17be013622132c5bcf4f465d1554a5513e7ce03c96bee85fc80"} Jul 20 05:09:21.228014 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.227820 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="156f6b7c1bc4b17be013622132c5bcf4f465d1554a5513e7ce03c96bee85fc80" Jul 20 05:09:21.364971 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.364935 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" Jul 20 05:09:21.403780 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.403745 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e040e05c-b612-48e9-a442-53b1505c7187-bundle\") pod \"e040e05c-b612-48e9-a442-53b1505c7187\" (UID: \"e040e05c-b612-48e9-a442-53b1505c7187\") " Jul 20 05:09:21.403994 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.403825 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e040e05c-b612-48e9-a442-53b1505c7187-util\") pod \"e040e05c-b612-48e9-a442-53b1505c7187\" (UID: \"e040e05c-b612-48e9-a442-53b1505c7187\") " Jul 20 05:09:21.403994 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.403846 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8kdm\" (UniqueName: \"kubernetes.io/projected/e040e05c-b612-48e9-a442-53b1505c7187-kube-api-access-k8kdm\") pod \"e040e05c-b612-48e9-a442-53b1505c7187\" (UID: \"e040e05c-b612-48e9-a442-53b1505c7187\") " Jul 20 05:09:21.404490 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.404421 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e040e05c-b612-48e9-a442-53b1505c7187-bundle" (OuterVolumeSpecName: "bundle") pod "e040e05c-b612-48e9-a442-53b1505c7187" (UID: "e040e05c-b612-48e9-a442-53b1505c7187"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:09:21.406523 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.406488 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e040e05c-b612-48e9-a442-53b1505c7187-kube-api-access-k8kdm" (OuterVolumeSpecName: "kube-api-access-k8kdm") pod "e040e05c-b612-48e9-a442-53b1505c7187" (UID: "e040e05c-b612-48e9-a442-53b1505c7187"). InnerVolumeSpecName "kube-api-access-k8kdm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:09:21.409839 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.409815 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e040e05c-b612-48e9-a442-53b1505c7187-util" (OuterVolumeSpecName: "util") pod "e040e05c-b612-48e9-a442-53b1505c7187" (UID: "e040e05c-b612-48e9-a442-53b1505c7187"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:09:21.505137 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.505097 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e040e05c-b612-48e9-a442-53b1505c7187-bundle\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:09:21.505137 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.505129 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e040e05c-b612-48e9-a442-53b1505c7187-util\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:09:21.505137 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:21.505139 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8kdm\" (UniqueName: \"kubernetes.io/projected/e040e05c-b612-48e9-a442-53b1505c7187-kube-api-access-k8kdm\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:09:22.235220 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:22.235184 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" event={"ID":"e040e05c-b612-48e9-a442-53b1505c7187","Type":"ContainerDied","Data":"c440fa8129ba740ad375ea957868a4712ee7fc40bb08aa77b311626fa9cac3b4"} Jul 20 05:09:22.235220 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:22.235225 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78" Jul 20 05:09:22.235631 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:22.235231 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c440fa8129ba740ad375ea957868a4712ee7fc40bb08aa77b311626fa9cac3b4" Jul 20 05:09:28.175905 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:28.175866 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:28.978131 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:28.978069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:32.430926 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.430877 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-plxhr"] Jul 20 05:09:32.431448 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431429 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e245d100-85e9-47d3-bda0-e7f8367b2ad7" containerName="pull" Jul 20 05:09:32.431517 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431449 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e245d100-85e9-47d3-bda0-e7f8367b2ad7" containerName="pull" Jul 20 05:09:32.431517 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431460 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e040e05c-b612-48e9-a442-53b1505c7187" containerName="extract" Jul 20 05:09:32.431517 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431470 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e040e05c-b612-48e9-a442-53b1505c7187" containerName="extract" Jul 20 05:09:32.431517 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431485 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47df011b-30b9-475e-8bdb-f28c2a1cfe8e" containerName="extract" Jul 20 05:09:32.431517 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431494 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="47df011b-30b9-475e-8bdb-f28c2a1cfe8e" containerName="extract" Jul 20 05:09:32.431517 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431507 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e040e05c-b612-48e9-a442-53b1505c7187" containerName="pull" Jul 20 05:09:32.431517 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431515 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e040e05c-b612-48e9-a442-53b1505c7187" containerName="pull" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431528 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a4ced7e-c8df-47fb-86c6-b65d31593b06" containerName="pull" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431536 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a4ced7e-c8df-47fb-86c6-b65d31593b06" containerName="pull" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431550 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e245d100-85e9-47d3-bda0-e7f8367b2ad7" containerName="util" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431560 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e245d100-85e9-47d3-bda0-e7f8367b2ad7" containerName="util" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431573 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47df011b-30b9-475e-8bdb-f28c2a1cfe8e" containerName="util" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431581 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="47df011b-30b9-475e-8bdb-f28c2a1cfe8e" containerName="util" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431590 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e040e05c-b612-48e9-a442-53b1505c7187" containerName="util" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431599 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e040e05c-b612-48e9-a442-53b1505c7187" containerName="util" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431620 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a4ced7e-c8df-47fb-86c6-b65d31593b06" containerName="extract" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431628 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a4ced7e-c8df-47fb-86c6-b65d31593b06" containerName="extract" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431639 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47df011b-30b9-475e-8bdb-f28c2a1cfe8e" containerName="pull" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431646 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="47df011b-30b9-475e-8bdb-f28c2a1cfe8e" containerName="pull" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431656 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e245d100-85e9-47d3-bda0-e7f8367b2ad7" containerName="extract" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431664 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e245d100-85e9-47d3-bda0-e7f8367b2ad7" containerName="extract" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431674 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a4ced7e-c8df-47fb-86c6-b65d31593b06" containerName="util" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431682 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a4ced7e-c8df-47fb-86c6-b65d31593b06" containerName="util" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431777 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a4ced7e-c8df-47fb-86c6-b65d31593b06" containerName="extract" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431794 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="47df011b-30b9-475e-8bdb-f28c2a1cfe8e" containerName="extract" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431804 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e040e05c-b612-48e9-a442-53b1505c7187" containerName="extract" Jul 20 05:09:32.431850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.431814 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e245d100-85e9-47d3-bda0-e7f8367b2ad7" containerName="extract" Jul 20 05:09:32.435377 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.435355 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-plxhr" Jul 20 05:09:32.438946 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.438924 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-n7cp2\"" Jul 20 05:09:32.449702 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.449676 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-plxhr"] Jul 20 05:09:32.509621 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.509578 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlppx\" (UniqueName: \"kubernetes.io/projected/5f06de2c-28ab-4315-8dc1-9aaa4e1a0f6f-kube-api-access-jlppx\") pod \"authorino-operator-75cc8d5cb7-plxhr\" (UID: \"5f06de2c-28ab-4315-8dc1-9aaa4e1a0f6f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-plxhr" Jul 20 05:09:32.610736 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.610690 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlppx\" (UniqueName: \"kubernetes.io/projected/5f06de2c-28ab-4315-8dc1-9aaa4e1a0f6f-kube-api-access-jlppx\") pod \"authorino-operator-75cc8d5cb7-plxhr\" (UID: \"5f06de2c-28ab-4315-8dc1-9aaa4e1a0f6f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-plxhr" Jul 20 05:09:32.620100 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.620066 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlppx\" (UniqueName: \"kubernetes.io/projected/5f06de2c-28ab-4315-8dc1-9aaa4e1a0f6f-kube-api-access-jlppx\") pod \"authorino-operator-75cc8d5cb7-plxhr\" (UID: \"5f06de2c-28ab-4315-8dc1-9aaa4e1a0f6f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-plxhr" Jul 20 05:09:32.745657 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.745636 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-plxhr" Jul 20 05:09:32.885439 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.885412 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-plxhr"] Jul 20 05:09:32.888077 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:09:32.888043 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f06de2c_28ab_4315_8dc1_9aaa4e1a0f6f.slice/crio-9264806d28c46c7bfedb4758fa74c1654a3f0185ff2d6de629cb8efc0d5ae848 WatchSource:0}: Error finding container 9264806d28c46c7bfedb4758fa74c1654a3f0185ff2d6de629cb8efc0d5ae848: Status 404 returned error can't find the container with id 9264806d28c46c7bfedb4758fa74c1654a3f0185ff2d6de629cb8efc0d5ae848 Jul 20 05:09:32.890319 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:32.890296 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 05:09:33.300648 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.300610 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-577bc74b9b-ht5q5"] Jul 20 05:09:33.305645 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.305610 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.306523 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.306478 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-plxhr" event={"ID":"5f06de2c-28ab-4315-8dc1-9aaa4e1a0f6f","Type":"ContainerStarted","Data":"9264806d28c46c7bfedb4758fa74c1654a3f0185ff2d6de629cb8efc0d5ae848"} Jul 20 05:09:33.309371 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.309157 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-25rk9\"" Jul 20 05:09:33.309371 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.309233 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 20 05:09:33.309371 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.309252 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 20 05:09:33.309371 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.309261 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 20 05:09:33.309371 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.309295 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 20 05:09:33.309371 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.309368 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 20 05:09:33.315122 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.315095 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 20 05:09:33.317895 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.317830 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-577bc74b9b-ht5q5"] Jul 20 05:09:33.363748 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.363710 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:33.418440 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.418407 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35c45dc4-778b-4721-8ecf-696db95c8dab-console-config\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.418440 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.418442 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35c45dc4-778b-4721-8ecf-696db95c8dab-console-serving-cert\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.418642 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.418478 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35c45dc4-778b-4721-8ecf-696db95c8dab-trusted-ca-bundle\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.418642 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.418502 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35c45dc4-778b-4721-8ecf-696db95c8dab-service-ca\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.418642 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.418534 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35c45dc4-778b-4721-8ecf-696db95c8dab-oauth-serving-cert\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.418642 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.418560 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcrj7\" (UniqueName: \"kubernetes.io/projected/35c45dc4-778b-4721-8ecf-696db95c8dab-kube-api-access-tcrj7\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.418642 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.418601 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35c45dc4-778b-4721-8ecf-696db95c8dab-console-oauth-config\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.465296 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.465263 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:33.519948 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.519895 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35c45dc4-778b-4721-8ecf-696db95c8dab-oauth-serving-cert\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.520148 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.519982 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcrj7\" (UniqueName: \"kubernetes.io/projected/35c45dc4-778b-4721-8ecf-696db95c8dab-kube-api-access-tcrj7\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.520148 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.520017 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35c45dc4-778b-4721-8ecf-696db95c8dab-console-oauth-config\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.520148 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.520107 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35c45dc4-778b-4721-8ecf-696db95c8dab-console-config\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.520313 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.520227 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35c45dc4-778b-4721-8ecf-696db95c8dab-console-serving-cert\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.520372 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.520307 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35c45dc4-778b-4721-8ecf-696db95c8dab-trusted-ca-bundle\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.520372 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.520340 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35c45dc4-778b-4721-8ecf-696db95c8dab-service-ca\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.520851 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.520795 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35c45dc4-778b-4721-8ecf-696db95c8dab-oauth-serving-cert\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.520851 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.520820 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35c45dc4-778b-4721-8ecf-696db95c8dab-console-config\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.521066 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.521040 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35c45dc4-778b-4721-8ecf-696db95c8dab-service-ca\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.521527 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.521472 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35c45dc4-778b-4721-8ecf-696db95c8dab-trusted-ca-bundle\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.524106 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.524059 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35c45dc4-778b-4721-8ecf-696db95c8dab-console-serving-cert\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.524554 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.524533 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35c45dc4-778b-4721-8ecf-696db95c8dab-console-oauth-config\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.546042 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.546008 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcrj7\" (UniqueName: \"kubernetes.io/projected/35c45dc4-778b-4721-8ecf-696db95c8dab-kube-api-access-tcrj7\") pod \"console-577bc74b9b-ht5q5\" (UID: \"35c45dc4-778b-4721-8ecf-696db95c8dab\") " pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.619083 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.618985 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:33.663851 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.663601 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:33.794877 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:33.794845 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-577bc74b9b-ht5q5"] Jul 20 05:09:33.797010 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:09:33.796975 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c45dc4_778b_4721_8ecf_696db95c8dab.slice/crio-4e2db6f2afbe536d2f76db877bcbfb27017a6e1d647f0df628f153acf140e2c5 WatchSource:0}: Error finding container 4e2db6f2afbe536d2f76db877bcbfb27017a6e1d647f0df628f153acf140e2c5: Status 404 returned error can't find the container with id 4e2db6f2afbe536d2f76db877bcbfb27017a6e1d647f0df628f153acf140e2c5 Jul 20 05:09:34.314740 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:34.314699 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-577bc74b9b-ht5q5" event={"ID":"35c45dc4-778b-4721-8ecf-696db95c8dab","Type":"ContainerStarted","Data":"ce418c35524484224e1e2899f4e2eb8079ce03175a95cadcc5d5cf270ef4b69e"} Jul 20 05:09:34.314910 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:34.314758 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-577bc74b9b-ht5q5" event={"ID":"35c45dc4-778b-4721-8ecf-696db95c8dab","Type":"ContainerStarted","Data":"4e2db6f2afbe536d2f76db877bcbfb27017a6e1d647f0df628f153acf140e2c5"} Jul 20 05:09:34.334399 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:34.334333 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-577bc74b9b-ht5q5" podStartSLOduration=1.33431025 podStartE2EDuration="1.33431025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:09:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:09:34.330265735 +0000 UTC m=+617.376792661" watchObservedRunningTime="2026-07-20 05:09:34.33431025 +0000 UTC m=+617.380837157" Jul 20 05:09:35.322196 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:35.322160 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-plxhr" event={"ID":"5f06de2c-28ab-4315-8dc1-9aaa4e1a0f6f","Type":"ContainerStarted","Data":"1034649f43fcf26b0707bdf7eaaa0eb063825838287ebb07e82943d2f4c6a0cb"} Jul 20 05:09:35.340723 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:35.340670 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-plxhr" podStartSLOduration=1.029255036 podStartE2EDuration="3.340653515s" totalImagesPullingTime="2.311398479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:09:35.338541806 +0000 UTC m=+618.385068738" watchObservedRunningTime="2026-07-20 05:09:35.340653515 +0000 UTC m=+618.387180415" Jul 20 05:09:35.466132 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:35.466049 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-k6jlw"] Jul 20 05:09:35.469743 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:35.469717 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k6jlw" Jul 20 05:09:35.472386 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:35.472364 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 20 05:09:35.472575 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:35.472553 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-sbp8m\"" Jul 20 05:09:35.485167 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:35.485134 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-k6jlw"] Jul 20 05:09:35.543515 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:35.543468 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw5qr\" (UniqueName: \"kubernetes.io/projected/66e63abf-0805-462f-8808-117879a8ad7b-kube-api-access-lw5qr\") pod \"dns-operator-controller-manager-864dd78c4c-k6jlw\" (UID: \"66e63abf-0805-462f-8808-117879a8ad7b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k6jlw" Jul 20 05:09:35.628181 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:35.628147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:35.644400 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:35.644359 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw5qr\" (UniqueName: \"kubernetes.io/projected/66e63abf-0805-462f-8808-117879a8ad7b-kube-api-access-lw5qr\") pod \"dns-operator-controller-manager-864dd78c4c-k6jlw\" (UID: \"66e63abf-0805-462f-8808-117879a8ad7b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k6jlw" Jul 20 05:09:35.653619 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:35.653589 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw5qr\" (UniqueName: \"kubernetes.io/projected/66e63abf-0805-462f-8808-117879a8ad7b-kube-api-access-lw5qr\") pod \"dns-operator-controller-manager-864dd78c4c-k6jlw\" (UID: \"66e63abf-0805-462f-8808-117879a8ad7b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k6jlw" Jul 20 05:09:35.781354 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:35.781312 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k6jlw" Jul 20 05:09:35.918156 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:35.918129 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-k6jlw"] Jul 20 05:09:35.920235 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:09:35.920197 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66e63abf_0805_462f_8808_117879a8ad7b.slice/crio-cf1115ecd2fc4f749e914f63bf1d3849b4768a17d31d0fa83b82d50f5798392c WatchSource:0}: Error finding container cf1115ecd2fc4f749e914f63bf1d3849b4768a17d31d0fa83b82d50f5798392c: Status 404 returned error can't find the container with id cf1115ecd2fc4f749e914f63bf1d3849b4768a17d31d0fa83b82d50f5798392c Jul 20 05:09:36.328947 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:36.328916 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k6jlw" event={"ID":"66e63abf-0805-462f-8808-117879a8ad7b","Type":"ContainerStarted","Data":"cf1115ecd2fc4f749e914f63bf1d3849b4768a17d31d0fa83b82d50f5798392c"} Jul 20 05:09:36.329375 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:36.329128 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-plxhr" Jul 20 05:09:36.790212 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:36.790178 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:37.617644 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:37.617564 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:38.341326 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:38.341288 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k6jlw" event={"ID":"66e63abf-0805-462f-8808-117879a8ad7b","Type":"ContainerStarted","Data":"6555cc76d304e6836070cde7231dd21cc0e5775061504e866e994d528eb2f308"} Jul 20 05:09:38.341515 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:38.341422 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k6jlw" Jul 20 05:09:38.360000 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:38.359919 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k6jlw" podStartSLOduration=1.366959599 podStartE2EDuration="3.359901125s" totalImagesPullingTime="1.992941526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:09:38.358245862 +0000 UTC m=+621.404772815" watchObservedRunningTime="2026-07-20 05:09:38.359901125 +0000 UTC m=+621.406428030" Jul 20 05:09:39.391638 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:39.391557 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:40.230338 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:40.230305 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:43.449107 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.449070 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz"] Jul 20 05:09:43.452859 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.452833 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:09:43.455686 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.455650 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9dxbz\"" Jul 20 05:09:43.464237 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.464207 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:43.466447 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.466419 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz"] Jul 20 05:09:43.615742 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.615704 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cx5jz\" (UID: \"1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:09:43.615923 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.615848 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptrr2\" (UniqueName: \"kubernetes.io/projected/1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b-kube-api-access-ptrr2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cx5jz\" (UID: \"1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:09:43.619619 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.619601 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:43.619676 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.619650 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:43.624359 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.624334 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:43.716775 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.716669 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cx5jz\" (UID: \"1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:09:43.717208 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.717174 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptrr2\" (UniqueName: \"kubernetes.io/projected/1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b-kube-api-access-ptrr2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cx5jz\" (UID: \"1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:09:43.717358 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.717245 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cx5jz\" (UID: \"1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:09:43.726697 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.726672 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptrr2\" (UniqueName: \"kubernetes.io/projected/1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b-kube-api-access-ptrr2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cx5jz\" (UID: \"1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:09:43.764013 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.763974 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:09:43.908427 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:43.908396 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz"] Jul 20 05:09:43.910433 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:09:43.910403 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fbd28de_2fb9_41eb_ad91_a2fc5dd4a13b.slice/crio-a9351695945c93080f7ec1565567fdd57cbf2a83fb4d60d79ba0f1160992b478 WatchSource:0}: Error finding container a9351695945c93080f7ec1565567fdd57cbf2a83fb4d60d79ba0f1160992b478: Status 404 returned error can't find the container with id a9351695945c93080f7ec1565567fdd57cbf2a83fb4d60d79ba0f1160992b478 Jul 20 05:09:44.376045 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:44.376011 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" event={"ID":"1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b","Type":"ContainerStarted","Data":"a9351695945c93080f7ec1565567fdd57cbf2a83fb4d60d79ba0f1160992b478"} Jul 20 05:09:44.379975 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:44.379937 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-577bc74b9b-ht5q5" Jul 20 05:09:44.568561 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:44.568526 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:44.799153 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:44.799119 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:44.897816 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:44.897779 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:45.394017 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:45.393977 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:45.497840 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:45.497802 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:45.791562 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:45.791526 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:45.891926 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:45.891891 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:47.337993 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:47.337947 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-plxhr" Jul 20 05:09:49.351237 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:49.351202 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k6jlw" Jul 20 05:09:49.404266 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:49.404219 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" event={"ID":"1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b","Type":"ContainerStarted","Data":"967e365ca50be1e1ad37b208733c8b774a2ead2727f58555ad853a4b99f44b6d"} Jul 20 05:09:49.404484 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:49.404414 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:09:49.404560 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:49.404522 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:49.422669 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:49.422619 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" podStartSLOduration=1.6798354149999999 podStartE2EDuration="6.422600331s" totalImagesPullingTime="4.742764916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:09:49.420996351 +0000 UTC m=+632.467523285" watchObservedRunningTime="2026-07-20 05:09:49.422600331 +0000 UTC m=+632.469127236" Jul 20 05:09:49.500574 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:49.500525 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:51.309595 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.309549 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-q2782"] Jul 20 05:09:51.314775 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.314752 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" Jul 20 05:09:51.317322 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.317299 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 20 05:09:51.317464 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.317342 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wx9pn\"" Jul 20 05:09:51.317464 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.317343 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 20 05:09:51.321301 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.321260 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:51.324007 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.323843 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-q2782"] Jul 20 05:09:51.496387 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.496343 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9646f70b-dda3-496b-98a6-6e8e33e617d8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-q2782\" (UID: \"9646f70b-dda3-496b-98a6-6e8e33e617d8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" Jul 20 05:09:51.496581 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.496478 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9646f70b-dda3-496b-98a6-6e8e33e617d8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-q2782\" (UID: \"9646f70b-dda3-496b-98a6-6e8e33e617d8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" Jul 20 05:09:51.496581 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.496550 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kpqd\" (UniqueName: \"kubernetes.io/projected/9646f70b-dda3-496b-98a6-6e8e33e617d8-kube-api-access-6kpqd\") pod \"kuadrant-console-plugin-66b5774985-q2782\" (UID: \"9646f70b-dda3-496b-98a6-6e8e33e617d8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" Jul 20 05:09:51.597257 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.597159 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9646f70b-dda3-496b-98a6-6e8e33e617d8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-q2782\" (UID: \"9646f70b-dda3-496b-98a6-6e8e33e617d8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" Jul 20 05:09:51.597257 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.597235 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kpqd\" (UniqueName: \"kubernetes.io/projected/9646f70b-dda3-496b-98a6-6e8e33e617d8-kube-api-access-6kpqd\") pod \"kuadrant-console-plugin-66b5774985-q2782\" (UID: \"9646f70b-dda3-496b-98a6-6e8e33e617d8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" Jul 20 05:09:51.597463 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.597282 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9646f70b-dda3-496b-98a6-6e8e33e617d8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-q2782\" (UID: \"9646f70b-dda3-496b-98a6-6e8e33e617d8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" Jul 20 05:09:51.598057 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.598038 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9646f70b-dda3-496b-98a6-6e8e33e617d8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-q2782\" (UID: \"9646f70b-dda3-496b-98a6-6e8e33e617d8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" Jul 20 05:09:51.599709 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.599686 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9646f70b-dda3-496b-98a6-6e8e33e617d8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-q2782\" (UID: \"9646f70b-dda3-496b-98a6-6e8e33e617d8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" Jul 20 05:09:51.605120 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.605099 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:51.606108 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.606089 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kpqd\" (UniqueName: \"kubernetes.io/projected/9646f70b-dda3-496b-98a6-6e8e33e617d8-kube-api-access-6kpqd\") pod \"kuadrant-console-plugin-66b5774985-q2782\" (UID: \"9646f70b-dda3-496b-98a6-6e8e33e617d8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" Jul 20 05:09:51.626799 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.626770 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" Jul 20 05:09:51.766486 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:51.766459 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-q2782"] Jul 20 05:09:51.767865 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:09:51.767838 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9646f70b_dda3_496b_98a6_6e8e33e617d8.slice/crio-431cf6de034bb59c744a814f454088de1a8f54f556e83c35449320d248554c5e WatchSource:0}: Error finding container 431cf6de034bb59c744a814f454088de1a8f54f556e83c35449320d248554c5e: Status 404 returned error can't find the container with id 431cf6de034bb59c744a814f454088de1a8f54f556e83c35449320d248554c5e Jul 20 05:09:52.177109 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:52.177072 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:52.421158 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:52.421121 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" event={"ID":"9646f70b-dda3-496b-98a6-6e8e33e617d8","Type":"ContainerStarted","Data":"431cf6de034bb59c744a814f454088de1a8f54f556e83c35449320d248554c5e"} Jul 20 05:09:53.216387 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:53.216348 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:53.319776 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:53.319732 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:56.726592 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:56.726550 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:09:56.827608 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:09:56.827567 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:00.412284 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:00.412249 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:10:02.123656 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.123612 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz"] Jul 20 05:10:02.124125 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.123814 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" podUID="1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b" containerName="manager" containerID="cri-o://967e365ca50be1e1ad37b208733c8b774a2ead2727f58555ad853a4b99f44b6d" gracePeriod=2 Jul 20 05:10:02.136154 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.136079 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz"] Jul 20 05:10:02.146766 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:10:02.146694 2576 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-cx5jz\": pods \"kuadrant-operator-controller-manager-8684c9ff49-cx5jz\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:10:02.152217 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.152184 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh"] Jul 20 05:10:02.152788 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.152767 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b" containerName="manager" Jul 20 05:10:02.152880 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.152791 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b" containerName="manager" Jul 20 05:10:02.152951 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.152934 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b" containerName="manager" Jul 20 05:10:02.196698 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.196666 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh"] Jul 20 05:10:02.196910 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.196801 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" Jul 20 05:10:02.199245 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:10:02.199196 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cx5jz\" is forbidden: User \"system:node:ip-10-0-128-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-33.ec2.internal' and this object" podUID="1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:10:02.201940 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.201916 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h26h\" (UniqueName: \"kubernetes.io/projected/6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c-kube-api-access-4h26h\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t87jh\" (UID: \"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" Jul 20 05:10:02.202089 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.202067 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t87jh\" (UID: \"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" Jul 20 05:10:02.303349 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.303310 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t87jh\" (UID: \"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" Jul 20 05:10:02.303527 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.303414 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h26h\" (UniqueName: \"kubernetes.io/projected/6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c-kube-api-access-4h26h\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t87jh\" (UID: \"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" Jul 20 05:10:02.303755 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.303734 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t87jh\" (UID: \"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" Jul 20 05:10:02.312431 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.312394 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h26h\" (UniqueName: \"kubernetes.io/projected/6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c-kube-api-access-4h26h\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t87jh\" (UID: \"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" Jul 20 05:10:02.509452 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:02.509410 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" Jul 20 05:10:04.774831 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:04.774790 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:04.876185 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:04.876105 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:05.163520 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:05.163415 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:05.264105 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:05.264070 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:08.166687 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:08.166651 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:08.214622 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:08.214535 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:11.774560 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:11.774520 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:11.875534 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:11.875503 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:12.887396 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:12.887368 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:10:12.889809 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:10:12.889776 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cx5jz\" is forbidden: User \"system:node:ip-10-0-128-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-33.ec2.internal' and this object" podUID="1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:10:12.892465 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:12.892443 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh"] Jul 20 05:10:12.896247 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:10:12.896221 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a1e2b8c_77ce_4994_bbbc_d6bb8ab1374c.slice/crio-f9cda68498f0238f37ea1c4c5e9f6dfede5cdc2498506d76a42ffe5b1f1e0a20 WatchSource:0}: Error finding container f9cda68498f0238f37ea1c4c5e9f6dfede5cdc2498506d76a42ffe5b1f1e0a20: Status 404 returned error can't find the container with id f9cda68498f0238f37ea1c4c5e9f6dfede5cdc2498506d76a42ffe5b1f1e0a20 Jul 20 05:10:13.024467 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.024376 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptrr2\" (UniqueName: \"kubernetes.io/projected/1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b-kube-api-access-ptrr2\") pod \"1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b\" (UID: \"1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b\") " Jul 20 05:10:13.024467 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.024443 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b-extensions-socket-volume\") pod \"1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b\" (UID: \"1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b\") " Jul 20 05:10:13.024977 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.024933 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b" (UID: "1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:10:13.026507 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.026481 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b-kube-api-access-ptrr2" (OuterVolumeSpecName: "kube-api-access-ptrr2") pod "1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b" (UID: "1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b"). InnerVolumeSpecName "kube-api-access-ptrr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:10:13.125895 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.125845 2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b-extensions-socket-volume\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:10:13.125895 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.125889 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptrr2\" (UniqueName: \"kubernetes.io/projected/1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b-kube-api-access-ptrr2\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:10:13.558423 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.558391 2576 generic.go:356] "Generic (PLEG): container finished" podID="1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b" containerID="967e365ca50be1e1ad37b208733c8b774a2ead2727f58555ad853a4b99f44b6d" exitCode=0 Jul 20 05:10:13.558626 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.558437 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:10:13.558626 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.558472 2576 scope.go:117] "RemoveContainer" containerID="967e365ca50be1e1ad37b208733c8b774a2ead2727f58555ad853a4b99f44b6d" Jul 20 05:10:13.560384 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.560354 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" event={"ID":"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c","Type":"ContainerStarted","Data":"e909d9a499e37f18d78cb688271184fa0f0704f906d1fec4714174afb10984fd"} Jul 20 05:10:13.560496 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.560407 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" event={"ID":"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c","Type":"ContainerStarted","Data":"f9cda68498f0238f37ea1c4c5e9f6dfede5cdc2498506d76a42ffe5b1f1e0a20"} Jul 20 05:10:13.560496 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.560437 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" Jul 20 05:10:13.561307 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:10:13.561275 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cx5jz\" is forbidden: User \"system:node:ip-10-0-128-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-33.ec2.internal' and this object" podUID="1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:10:13.562123 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.562096 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" event={"ID":"9646f70b-dda3-496b-98a6-6e8e33e617d8","Type":"ContainerStarted","Data":"a40972a98220a0fd6946ad3293de4abe0e50ad565282cdce6a37bba02c80b6e5"} Jul 20 05:10:13.568204 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.568189 2576 scope.go:117] "RemoveContainer" containerID="967e365ca50be1e1ad37b208733c8b774a2ead2727f58555ad853a4b99f44b6d" Jul 20 05:10:13.568475 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:10:13.568457 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"967e365ca50be1e1ad37b208733c8b774a2ead2727f58555ad853a4b99f44b6d\": container with ID starting with 967e365ca50be1e1ad37b208733c8b774a2ead2727f58555ad853a4b99f44b6d not found: ID does not exist" containerID="967e365ca50be1e1ad37b208733c8b774a2ead2727f58555ad853a4b99f44b6d" Jul 20 05:10:13.568521 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.568484 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"967e365ca50be1e1ad37b208733c8b774a2ead2727f58555ad853a4b99f44b6d"} err="failed to get container status \"967e365ca50be1e1ad37b208733c8b774a2ead2727f58555ad853a4b99f44b6d\": rpc error: code = NotFound desc = could not find container \"967e365ca50be1e1ad37b208733c8b774a2ead2727f58555ad853a4b99f44b6d\": container with ID starting with 967e365ca50be1e1ad37b208733c8b774a2ead2727f58555ad853a4b99f44b6d not found: ID does not exist" Jul 20 05:10:13.580410 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:10:13.580378 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cx5jz\" is forbidden: User \"system:node:ip-10-0-128-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-33.ec2.internal' and this object" podUID="1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:10:13.581108 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.581070 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" podStartSLOduration=11.581057068 podStartE2EDuration="11.581057068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:10:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:10:13.57838138 +0000 UTC m=+656.624908285" watchObservedRunningTime="2026-07-20 05:10:13.581057068 +0000 UTC m=+656.627583973" Jul 20 05:10:13.597125 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.597085 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2782" podStartSLOduration=1.511718636 podStartE2EDuration="22.597071373s" totalImagesPullingTime="21.085352737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:10:13.596114371 +0000 UTC m=+656.642641279" watchObservedRunningTime="2026-07-20 05:10:13.597071373 +0000 UTC m=+656.643598279" Jul 20 05:10:13.598054 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:10:13.598017 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cx5jz\" is forbidden: User \"system:node:ip-10-0-128-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-33.ec2.internal' and this object" podUID="1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cx5jz" Jul 20 05:10:13.654784 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:13.654751 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b" path="/var/lib/kubelet/pods/1fbd28de-2fb9-41eb-ad91-a2fc5dd4a13b/volumes" Jul 20 05:10:15.306740 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:15.306700 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:15.407079 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:15.407023 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:18.038924 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.038885 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh"] Jul 20 05:10:18.039343 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.039080 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" podUID="6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c" containerName="manager" containerID="cri-o://e909d9a499e37f18d78cb688271184fa0f0704f906d1fec4714174afb10984fd" gracePeriod=10 Jul 20 05:10:18.040880 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.040855 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" Jul 20 05:10:18.294778 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.294705 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" Jul 20 05:10:18.373328 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.373288 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4h26h\" (UniqueName: \"kubernetes.io/projected/6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c-kube-api-access-4h26h\") pod \"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c\" (UID: \"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c\") " Jul 20 05:10:18.373508 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.373432 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c-extensions-socket-volume\") pod \"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c\" (UID: \"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c\") " Jul 20 05:10:18.373837 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.373814 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c" (UID: "6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:10:18.375387 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.375361 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c-kube-api-access-4h26h" (OuterVolumeSpecName: "kube-api-access-4h26h") pod "6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c" (UID: "6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c"). InnerVolumeSpecName "kube-api-access-4h26h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:10:18.474902 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.474859 2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c-extensions-socket-volume\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:10:18.474902 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.474897 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h26h\" (UniqueName: \"kubernetes.io/projected/6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c-kube-api-access-4h26h\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:10:18.592913 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.592812 2576 generic.go:356] "Generic (PLEG): container finished" podID="6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c" containerID="e909d9a499e37f18d78cb688271184fa0f0704f906d1fec4714174afb10984fd" exitCode=0 Jul 20 05:10:18.592913 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.592861 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" Jul 20 05:10:18.592913 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.592889 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" event={"ID":"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c","Type":"ContainerDied","Data":"e909d9a499e37f18d78cb688271184fa0f0704f906d1fec4714174afb10984fd"} Jul 20 05:10:18.593206 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.592942 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh" event={"ID":"6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c","Type":"ContainerDied","Data":"f9cda68498f0238f37ea1c4c5e9f6dfede5cdc2498506d76a42ffe5b1f1e0a20"} Jul 20 05:10:18.593206 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.592976 2576 scope.go:117] "RemoveContainer" containerID="e909d9a499e37f18d78cb688271184fa0f0704f906d1fec4714174afb10984fd" Jul 20 05:10:18.602923 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.602906 2576 scope.go:117] "RemoveContainer" containerID="e909d9a499e37f18d78cb688271184fa0f0704f906d1fec4714174afb10984fd" Jul 20 05:10:18.603236 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:10:18.603218 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e909d9a499e37f18d78cb688271184fa0f0704f906d1fec4714174afb10984fd\": container with ID starting with e909d9a499e37f18d78cb688271184fa0f0704f906d1fec4714174afb10984fd not found: ID does not exist" containerID="e909d9a499e37f18d78cb688271184fa0f0704f906d1fec4714174afb10984fd" Jul 20 05:10:18.603310 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.603244 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e909d9a499e37f18d78cb688271184fa0f0704f906d1fec4714174afb10984fd"} err="failed to get container status \"e909d9a499e37f18d78cb688271184fa0f0704f906d1fec4714174afb10984fd\": rpc error: code = NotFound desc = could not find container \"e909d9a499e37f18d78cb688271184fa0f0704f906d1fec4714174afb10984fd\": container with ID starting with e909d9a499e37f18d78cb688271184fa0f0704f906d1fec4714174afb10984fd not found: ID does not exist" Jul 20 05:10:18.615254 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.615228 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh"] Jul 20 05:10:18.620867 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:18.620845 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t87jh"] Jul 20 05:10:19.652837 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:19.652794 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c" path="/var/lib/kubelet/pods/6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c/volumes" Jul 20 05:10:19.746489 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:19.746455 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:19.847094 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:19.847060 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:27.061724 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:27.061686 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:34.363460 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.363423 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh"] Jul 20 05:10:34.364109 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.364086 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c" containerName="manager" Jul 20 05:10:34.364164 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.364113 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c" containerName="manager" Jul 20 05:10:34.364218 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.364206 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a1e2b8c-77ce-4994-bbbc-d6bb8ab1374c" containerName="manager" Jul 20 05:10:34.389018 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.388987 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:34.535823 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.535785 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:34.578583 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.578543 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh"] Jul 20 05:10:34.578754 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.578661 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.581598 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.581573 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-k64mk\"" Jul 20 05:10:34.728571 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.728486 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvzn2\" (UniqueName: \"kubernetes.io/projected/88b632e5-2858-48b7-817a-5704fe6f3469-kube-api-access-mvzn2\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.728571 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.728526 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88b632e5-2858-48b7-817a-5704fe6f3469-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.728571 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.728547 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.728852 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.728618 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.728852 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.728646 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88b632e5-2858-48b7-817a-5704fe6f3469-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.728852 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.728685 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88b632e5-2858-48b7-817a-5704fe6f3469-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.728852 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.728715 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.728852 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.728764 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.728852 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.728797 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.829460 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.829418 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88b632e5-2858-48b7-817a-5704fe6f3469-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.829460 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.829463 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.829684 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.829490 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.829684 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.829540 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.829684 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.829641 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvzn2\" (UniqueName: \"kubernetes.io/projected/88b632e5-2858-48b7-817a-5704fe6f3469-kube-api-access-mvzn2\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.829684 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.829662 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88b632e5-2858-48b7-817a-5704fe6f3469-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.829898 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.829688 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.829898 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.829743 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.829898 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.829767 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88b632e5-2858-48b7-817a-5704fe6f3469-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.830097 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.829901 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.830097 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.829910 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.830261 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.830224 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88b632e5-2858-48b7-817a-5704fe6f3469-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.830261 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.830236 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.830374 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.830258 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.832083 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.832057 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88b632e5-2858-48b7-817a-5704fe6f3469-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.832373 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.832350 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88b632e5-2858-48b7-817a-5704fe6f3469-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.837449 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.837425 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88b632e5-2858-48b7-817a-5704fe6f3469-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.837781 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.837760 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvzn2\" (UniqueName: \"kubernetes.io/projected/88b632e5-2858-48b7-817a-5704fe6f3469-kube-api-access-mvzn2\") pod \"maas-default-gateway-openshift-default-56654dcf7-r4zbh\" (UID: \"88b632e5-2858-48b7-817a-5704fe6f3469\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:34.889534 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:34.889495 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:35.066506 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:35.066474 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh"] Jul 20 05:10:35.068222 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:10:35.068190 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88b632e5_2858_48b7_817a_5704fe6f3469.slice/crio-8a2288d48440959b231b44d6826b671117381d8d8eee736dd1a3491e3f24972e WatchSource:0}: Error finding container 8a2288d48440959b231b44d6826b671117381d8d8eee736dd1a3491e3f24972e: Status 404 returned error can't find the container with id 8a2288d48440959b231b44d6826b671117381d8d8eee736dd1a3491e3f24972e Jul 20 05:10:35.070475 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:35.070436 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:10:35.070568 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:35.070500 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:10:35.070568 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:35.070529 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:10:35.689035 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:35.688989 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" event={"ID":"88b632e5-2858-48b7-817a-5704fe6f3469","Type":"ContainerStarted","Data":"32b757c171c2c73ec412c9331af2725dd75a7f6b892fcc508d770e07906ba586"} Jul 20 05:10:35.689430 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:35.689058 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" event={"ID":"88b632e5-2858-48b7-817a-5704fe6f3469","Type":"ContainerStarted","Data":"8a2288d48440959b231b44d6826b671117381d8d8eee736dd1a3491e3f24972e"} Jul 20 05:10:35.707940 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:35.707890 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" podStartSLOduration=1.707859105 podStartE2EDuration="1.707859105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:10:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:10:35.706507139 +0000 UTC m=+678.753034083" watchObservedRunningTime="2026-07-20 05:10:35.707859105 +0000 UTC m=+678.754386011" Jul 20 05:10:35.890580 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:35.890547 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:36.895478 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:36.895449 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:37.700379 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:37.700343 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:37.701481 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:37.701453 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:10:52.216678 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.216634 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lth87"] Jul 20 05:10:52.219065 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.219010 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:52.243064 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.243027 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lth87"] Jul 20 05:10:52.243229 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.243137 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" Jul 20 05:10:52.246267 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.246163 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 20 05:10:52.283086 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.283051 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46183638-a0ea-4935-8e23-b791f52eb0b9-config-file\") pod \"limitador-limitador-64c94fd64f-lth87\" (UID: \"46183638-a0ea-4935-8e23-b791f52eb0b9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" Jul 20 05:10:52.283258 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.283102 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrxht\" (UniqueName: \"kubernetes.io/projected/46183638-a0ea-4935-8e23-b791f52eb0b9-kube-api-access-nrxht\") pod \"limitador-limitador-64c94fd64f-lth87\" (UID: \"46183638-a0ea-4935-8e23-b791f52eb0b9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" Jul 20 05:10:52.314019 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.313984 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lth87"] Jul 20 05:10:52.318238 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.318208 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:52.370663 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.370617 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:52.383846 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.383820 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46183638-a0ea-4935-8e23-b791f52eb0b9-config-file\") pod \"limitador-limitador-64c94fd64f-lth87\" (UID: \"46183638-a0ea-4935-8e23-b791f52eb0b9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" Jul 20 05:10:52.383947 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.383852 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrxht\" (UniqueName: \"kubernetes.io/projected/46183638-a0ea-4935-8e23-b791f52eb0b9-kube-api-access-nrxht\") pod \"limitador-limitador-64c94fd64f-lth87\" (UID: \"46183638-a0ea-4935-8e23-b791f52eb0b9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" Jul 20 05:10:52.384440 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.384422 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46183638-a0ea-4935-8e23-b791f52eb0b9-config-file\") pod \"limitador-limitador-64c94fd64f-lth87\" (UID: \"46183638-a0ea-4935-8e23-b791f52eb0b9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" Jul 20 05:10:52.391637 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.391613 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrxht\" (UniqueName: \"kubernetes.io/projected/46183638-a0ea-4935-8e23-b791f52eb0b9-kube-api-access-nrxht\") pod \"limitador-limitador-64c94fd64f-lth87\" (UID: \"46183638-a0ea-4935-8e23-b791f52eb0b9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" Jul 20 05:10:52.553901 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.553846 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" Jul 20 05:10:52.692346 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.692312 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lth87"] Jul 20 05:10:52.693793 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:10:52.693767 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46183638_a0ea_4935_8e23_b791f52eb0b9.slice/crio-b23453d4cc59b6535fc260074b952d450652657f888690dfc1933fef876249fb WatchSource:0}: Error finding container b23453d4cc59b6535fc260074b952d450652657f888690dfc1933fef876249fb: Status 404 returned error can't find the container with id b23453d4cc59b6535fc260074b952d450652657f888690dfc1933fef876249fb Jul 20 05:10:52.787337 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:52.787304 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" event={"ID":"46183638-a0ea-4935-8e23-b791f52eb0b9","Type":"ContainerStarted","Data":"b23453d4cc59b6535fc260074b952d450652657f888690dfc1933fef876249fb"} Jul 20 05:10:53.040489 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:53.040453 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:53.093473 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:53.093426 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:10:55.807952 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:55.807914 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" event={"ID":"46183638-a0ea-4935-8e23-b791f52eb0b9","Type":"ContainerStarted","Data":"04ff949b9e0856466f578ea91fc5626a10edf0b7c2db548fd94cb1b9e130b890"} Jul 20 05:10:55.808340 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:55.808110 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" Jul 20 05:10:55.825273 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:10:55.825221 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" podStartSLOduration=1.06435986 podStartE2EDuration="3.825206849s" totalImagesPullingTime="2.760846989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:10:55.822342203 +0000 UTC m=+698.868869130" watchObservedRunningTime="2026-07-20 05:10:55.825206849 +0000 UTC m=+698.871733755" Jul 20 05:11:03.075987 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:03.075926 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:11:06.814919 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:06.814888 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" Jul 20 05:11:07.259758 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.259724 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lth87"] Jul 20 05:11:07.259936 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.259904 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" podUID="46183638-a0ea-4935-8e23-b791f52eb0b9" containerName="limitador" containerID="cri-o://04ff949b9e0856466f578ea91fc5626a10edf0b7c2db548fd94cb1b9e130b890" gracePeriod=30 Jul 20 05:11:07.821452 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.821396 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" Jul 20 05:11:07.872682 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.872650 2576 generic.go:356] "Generic (PLEG): container finished" podID="46183638-a0ea-4935-8e23-b791f52eb0b9" containerID="04ff949b9e0856466f578ea91fc5626a10edf0b7c2db548fd94cb1b9e130b890" exitCode=0 Jul 20 05:11:07.872856 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.872690 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" Jul 20 05:11:07.872856 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.872737 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" event={"ID":"46183638-a0ea-4935-8e23-b791f52eb0b9","Type":"ContainerDied","Data":"04ff949b9e0856466f578ea91fc5626a10edf0b7c2db548fd94cb1b9e130b890"} Jul 20 05:11:07.872856 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.872792 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lth87" event={"ID":"46183638-a0ea-4935-8e23-b791f52eb0b9","Type":"ContainerDied","Data":"b23453d4cc59b6535fc260074b952d450652657f888690dfc1933fef876249fb"} Jul 20 05:11:07.872856 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.872816 2576 scope.go:117] "RemoveContainer" containerID="04ff949b9e0856466f578ea91fc5626a10edf0b7c2db548fd94cb1b9e130b890" Jul 20 05:11:07.881644 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.881630 2576 scope.go:117] "RemoveContainer" containerID="04ff949b9e0856466f578ea91fc5626a10edf0b7c2db548fd94cb1b9e130b890" Jul 20 05:11:07.881855 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:11:07.881839 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ff949b9e0856466f578ea91fc5626a10edf0b7c2db548fd94cb1b9e130b890\": container with ID starting with 04ff949b9e0856466f578ea91fc5626a10edf0b7c2db548fd94cb1b9e130b890 not found: ID does not exist" containerID="04ff949b9e0856466f578ea91fc5626a10edf0b7c2db548fd94cb1b9e130b890" Jul 20 05:11:07.881897 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.881864 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ff949b9e0856466f578ea91fc5626a10edf0b7c2db548fd94cb1b9e130b890"} err="failed to get container status \"04ff949b9e0856466f578ea91fc5626a10edf0b7c2db548fd94cb1b9e130b890\": rpc error: code = NotFound desc = could not find container \"04ff949b9e0856466f578ea91fc5626a10edf0b7c2db548fd94cb1b9e130b890\": container with ID starting with 04ff949b9e0856466f578ea91fc5626a10edf0b7c2db548fd94cb1b9e130b890 not found: ID does not exist" Jul 20 05:11:07.914322 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.914302 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46183638-a0ea-4935-8e23-b791f52eb0b9-config-file\") pod \"46183638-a0ea-4935-8e23-b791f52eb0b9\" (UID: \"46183638-a0ea-4935-8e23-b791f52eb0b9\") " Jul 20 05:11:07.914412 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.914356 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrxht\" (UniqueName: \"kubernetes.io/projected/46183638-a0ea-4935-8e23-b791f52eb0b9-kube-api-access-nrxht\") pod \"46183638-a0ea-4935-8e23-b791f52eb0b9\" (UID: \"46183638-a0ea-4935-8e23-b791f52eb0b9\") " Jul 20 05:11:07.914619 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.914601 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46183638-a0ea-4935-8e23-b791f52eb0b9-config-file" (OuterVolumeSpecName: "config-file") pod "46183638-a0ea-4935-8e23-b791f52eb0b9" (UID: "46183638-a0ea-4935-8e23-b791f52eb0b9"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:11:07.914688 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.914676 2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46183638-a0ea-4935-8e23-b791f52eb0b9-config-file\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:11:07.916275 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:07.916257 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46183638-a0ea-4935-8e23-b791f52eb0b9-kube-api-access-nrxht" (OuterVolumeSpecName: "kube-api-access-nrxht") pod "46183638-a0ea-4935-8e23-b791f52eb0b9" (UID: "46183638-a0ea-4935-8e23-b791f52eb0b9"). InnerVolumeSpecName "kube-api-access-nrxht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:11:08.015511 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:08.015486 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrxht\" (UniqueName: \"kubernetes.io/projected/46183638-a0ea-4935-8e23-b791f52eb0b9-kube-api-access-nrxht\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:11:08.194655 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:08.194627 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lth87"] Jul 20 05:11:08.198390 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:08.198364 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lth87"] Jul 20 05:11:09.653692 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:09.653621 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46183638-a0ea-4935-8e23-b791f52eb0b9" path="/var/lib/kubelet/pods/46183638-a0ea-4935-8e23-b791f52eb0b9/volumes" Jul 20 05:11:13.264576 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.264540 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:11:13.363905 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.363875 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:11:13.706743 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.706665 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6"] Jul 20 05:11:13.707056 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.707043 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46183638-a0ea-4935-8e23-b791f52eb0b9" containerName="limitador" Jul 20 05:11:13.707103 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.707058 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="46183638-a0ea-4935-8e23-b791f52eb0b9" containerName="limitador" Jul 20 05:11:13.707143 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.707136 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="46183638-a0ea-4935-8e23-b791f52eb0b9" containerName="limitador" Jul 20 05:11:13.710209 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.710189 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6" Jul 20 05:11:13.712840 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.712818 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 20 05:11:13.712974 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.712901 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 20 05:11:13.713920 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.713902 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-64s85\"" Jul 20 05:11:13.714042 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.713904 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 20 05:11:13.718087 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.718068 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6"] Jul 20 05:11:13.735693 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.735653 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:11:13.865444 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.865413 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d05b2693-721a-4b8b-99e7-c439a57ea19d-data\") pod \"postgres-69fb4594cf-jmkg6\" (UID: \"d05b2693-721a-4b8b-99e7-c439a57ea19d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6" Jul 20 05:11:13.865616 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.865545 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvksk\" (UniqueName: \"kubernetes.io/projected/d05b2693-721a-4b8b-99e7-c439a57ea19d-kube-api-access-jvksk\") pod \"postgres-69fb4594cf-jmkg6\" (UID: \"d05b2693-721a-4b8b-99e7-c439a57ea19d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6" Jul 20 05:11:13.966489 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.966399 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvksk\" (UniqueName: \"kubernetes.io/projected/d05b2693-721a-4b8b-99e7-c439a57ea19d-kube-api-access-jvksk\") pod \"postgres-69fb4594cf-jmkg6\" (UID: \"d05b2693-721a-4b8b-99e7-c439a57ea19d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6" Jul 20 05:11:13.966489 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.966456 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d05b2693-721a-4b8b-99e7-c439a57ea19d-data\") pod \"postgres-69fb4594cf-jmkg6\" (UID: \"d05b2693-721a-4b8b-99e7-c439a57ea19d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6" Jul 20 05:11:13.966769 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.966752 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d05b2693-721a-4b8b-99e7-c439a57ea19d-data\") pod \"postgres-69fb4594cf-jmkg6\" (UID: \"d05b2693-721a-4b8b-99e7-c439a57ea19d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6" Jul 20 05:11:13.975230 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:13.975203 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvksk\" (UniqueName: \"kubernetes.io/projected/d05b2693-721a-4b8b-99e7-c439a57ea19d-kube-api-access-jvksk\") pod \"postgres-69fb4594cf-jmkg6\" (UID: \"d05b2693-721a-4b8b-99e7-c439a57ea19d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6" Jul 20 05:11:14.024311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:14.024285 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6" Jul 20 05:11:14.154737 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:14.154711 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6"] Jul 20 05:11:14.156519 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:11:14.156491 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd05b2693_721a_4b8b_99e7_c439a57ea19d.slice/crio-67ab3788d3c34c9d9c895136f1060d68b850612d9aa2ee0d06e7f0583a024b06 WatchSource:0}: Error finding container 67ab3788d3c34c9d9c895136f1060d68b850612d9aa2ee0d06e7f0583a024b06: Status 404 returned error can't find the container with id 67ab3788d3c34c9d9c895136f1060d68b850612d9aa2ee0d06e7f0583a024b06 Jul 20 05:11:14.919446 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:14.919373 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6" event={"ID":"d05b2693-721a-4b8b-99e7-c439a57ea19d","Type":"ContainerStarted","Data":"67ab3788d3c34c9d9c895136f1060d68b850612d9aa2ee0d06e7f0583a024b06"} Jul 20 05:11:18.797989 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:18.797947 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 20 05:11:18.946378 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:18.946337 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6" event={"ID":"d05b2693-721a-4b8b-99e7-c439a57ea19d","Type":"ContainerStarted","Data":"556438c3f5ada88af2500f8f28b8e48fa22c07e9f69968f6de26dcb9d8a4df6f"} Jul 20 05:11:18.946530 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:18.946397 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6" Jul 20 05:11:18.963051 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:18.963003 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6" podStartSLOduration=1.32546808 podStartE2EDuration="5.962986057s" totalImagesPullingTime="4.637517977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:11:18.960845337 +0000 UTC m=+722.007372243" watchObservedRunningTime="2026-07-20 05:11:18.962986057 +0000 UTC m=+722.009512957" Jul 20 05:11:24.982423 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:24.982395 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jmkg6" Jul 20 05:11:25.345248 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:25.345167 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:11:25.539933 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:25.539904 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:11:32.017695 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.017655 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:11:32.031763 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.031731 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59db86fcdd-dmqjt"] Jul 20 05:11:32.059306 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.059277 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59db86fcdd-dmqjt"] Jul 20 05:11:32.059460 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.059374 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59db86fcdd-dmqjt" Jul 20 05:11:32.061882 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.061852 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 20 05:11:32.062121 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.062105 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lqrrn\"" Jul 20 05:11:32.062196 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.062170 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 20 05:11:32.119891 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.119855 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:11:32.146371 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.146338 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5184d014-fc82-4576-8df6-85d71543c1d1-webhook-cert\") pod \"maas-controller-59db86fcdd-dmqjt\" (UID: \"5184d014-fc82-4576-8df6-85d71543c1d1\") " pod="opendatahub/maas-controller-59db86fcdd-dmqjt" Jul 20 05:11:32.146489 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.146396 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwhqg\" (UniqueName: \"kubernetes.io/projected/5184d014-fc82-4576-8df6-85d71543c1d1-kube-api-access-kwhqg\") pod \"maas-controller-59db86fcdd-dmqjt\" (UID: \"5184d014-fc82-4576-8df6-85d71543c1d1\") " pod="opendatahub/maas-controller-59db86fcdd-dmqjt" Jul 20 05:11:32.187618 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.187585 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:11:32.192652 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.192629 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-n2qnp"] Jul 20 05:11:32.196262 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.196247 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" Jul 20 05:11:32.205901 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.205878 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-n2qnp"] Jul 20 05:11:32.246408 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.246373 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:11:32.247467 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.247442 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4-webhook-cert\") pod \"maas-controller-6b4f66bf88-n2qnp\" (UID: \"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4\") " pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" Jul 20 05:11:32.247569 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.247492 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5184d014-fc82-4576-8df6-85d71543c1d1-webhook-cert\") pod \"maas-controller-59db86fcdd-dmqjt\" (UID: \"5184d014-fc82-4576-8df6-85d71543c1d1\") " pod="opendatahub/maas-controller-59db86fcdd-dmqjt" Jul 20 05:11:32.247569 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.247520 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwhqg\" (UniqueName: \"kubernetes.io/projected/5184d014-fc82-4576-8df6-85d71543c1d1-kube-api-access-kwhqg\") pod \"maas-controller-59db86fcdd-dmqjt\" (UID: \"5184d014-fc82-4576-8df6-85d71543c1d1\") " pod="opendatahub/maas-controller-59db86fcdd-dmqjt" Jul 20 05:11:32.247569 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.247537 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rmxv\" (UniqueName: \"kubernetes.io/projected/5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4-kube-api-access-4rmxv\") pod \"maas-controller-6b4f66bf88-n2qnp\" (UID: \"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4\") " pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" Jul 20 05:11:32.249861 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.249838 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5184d014-fc82-4576-8df6-85d71543c1d1-webhook-cert\") pod \"maas-controller-59db86fcdd-dmqjt\" (UID: \"5184d014-fc82-4576-8df6-85d71543c1d1\") " pod="opendatahub/maas-controller-59db86fcdd-dmqjt" Jul 20 05:11:32.254842 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.254824 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwhqg\" (UniqueName: \"kubernetes.io/projected/5184d014-fc82-4576-8df6-85d71543c1d1-kube-api-access-kwhqg\") pod \"maas-controller-59db86fcdd-dmqjt\" (UID: \"5184d014-fc82-4576-8df6-85d71543c1d1\") " pod="opendatahub/maas-controller-59db86fcdd-dmqjt" Jul 20 05:11:32.348662 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.348573 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4-webhook-cert\") pod \"maas-controller-6b4f66bf88-n2qnp\" (UID: \"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4\") " pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" Jul 20 05:11:32.348828 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.348675 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rmxv\" (UniqueName: \"kubernetes.io/projected/5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4-kube-api-access-4rmxv\") pod \"maas-controller-6b4f66bf88-n2qnp\" (UID: \"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4\") " pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" Jul 20 05:11:32.350920 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.350897 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4-webhook-cert\") pod \"maas-controller-6b4f66bf88-n2qnp\" (UID: \"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4\") " pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" Jul 20 05:11:32.357112 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.357088 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rmxv\" (UniqueName: \"kubernetes.io/projected/5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4-kube-api-access-4rmxv\") pod \"maas-controller-6b4f66bf88-n2qnp\" (UID: \"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4\") " pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" Jul 20 05:11:32.369855 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.369836 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59db86fcdd-dmqjt" Jul 20 05:11:32.503135 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.503110 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59db86fcdd-dmqjt"] Jul 20 05:11:32.504827 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:11:32.504799 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5184d014_fc82_4576_8df6_85d71543c1d1.slice/crio-34de34526f550f3f6aa581e080c52ad09c5e72c4aec148b5168084b7d82ae801 WatchSource:0}: Error finding container 34de34526f550f3f6aa581e080c52ad09c5e72c4aec148b5168084b7d82ae801: Status 404 returned error can't find the container with id 34de34526f550f3f6aa581e080c52ad09c5e72c4aec148b5168084b7d82ae801 Jul 20 05:11:32.507610 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.507591 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" Jul 20 05:11:32.640557 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:32.640383 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-n2qnp"] Jul 20 05:11:32.643267 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:11:32.643238 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb7e7a4_9e49_4a5d_a08b_d0620e5753a4.slice/crio-0788e1689cd0b2d76d9a2b2fce1a656306c3d755a262e87e4362eb09095a6dd1 WatchSource:0}: Error finding container 0788e1689cd0b2d76d9a2b2fce1a656306c3d755a262e87e4362eb09095a6dd1: Status 404 returned error can't find the container with id 0788e1689cd0b2d76d9a2b2fce1a656306c3d755a262e87e4362eb09095a6dd1 Jul 20 05:11:33.025832 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:33.025800 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59db86fcdd-dmqjt" event={"ID":"5184d014-fc82-4576-8df6-85d71543c1d1","Type":"ContainerStarted","Data":"34de34526f550f3f6aa581e080c52ad09c5e72c4aec148b5168084b7d82ae801"} Jul 20 05:11:33.027014 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:33.026988 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" event={"ID":"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4","Type":"ContainerStarted","Data":"0788e1689cd0b2d76d9a2b2fce1a656306c3d755a262e87e4362eb09095a6dd1"} Jul 20 05:11:35.197561 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:35.197523 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:11:35.295452 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:35.295365 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:11:36.048351 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:36.048311 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59db86fcdd-dmqjt" event={"ID":"5184d014-fc82-4576-8df6-85d71543c1d1","Type":"ContainerStarted","Data":"307f38e90667b6f1b9c13a9fb9b16b1f4c21d7f0aa4f98f4d8d1bb4029ad0ff2"} Jul 20 05:11:36.048539 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:36.048479 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59db86fcdd-dmqjt" Jul 20 05:11:36.049916 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:36.049887 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" event={"ID":"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4","Type":"ContainerStarted","Data":"56102ca47fd499b8e1cd2510afad5588f61f26024533e3c20750617ce7c03a89"} Jul 20 05:11:36.050049 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:36.049992 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" Jul 20 05:11:36.066894 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:36.066843 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59db86fcdd-dmqjt" podStartSLOduration=1.5514758400000002 podStartE2EDuration="4.066825881s" totalImagesPullingTime="2.515350041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:11:36.065723115 +0000 UTC m=+739.112250032" watchObservedRunningTime="2026-07-20 05:11:36.066825881 +0000 UTC m=+739.113352789" Jul 20 05:11:36.082665 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:36.082625 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" podStartSLOduration=1.699063515 podStartE2EDuration="4.082612087s" totalImagesPullingTime="2.383548572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:11:36.079994771 +0000 UTC m=+739.126521697" watchObservedRunningTime="2026-07-20 05:11:36.082612087 +0000 UTC m=+739.129138993" Jul 20 05:11:47.058898 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:47.058867 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59db86fcdd-dmqjt" Jul 20 05:11:47.059373 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:47.058935 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" Jul 20 05:11:47.129590 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:47.129558 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59db86fcdd-dmqjt"] Jul 20 05:11:47.130154 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:47.130119 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-59db86fcdd-dmqjt" podUID="5184d014-fc82-4576-8df6-85d71543c1d1" containerName="manager" containerID="cri-o://307f38e90667b6f1b9c13a9fb9b16b1f4c21d7f0aa4f98f4d8d1bb4029ad0ff2" gracePeriod=30 Jul 20 05:11:47.375254 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:47.375232 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59db86fcdd-dmqjt" Jul 20 05:11:47.488553 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:47.488525 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5184d014-fc82-4576-8df6-85d71543c1d1-webhook-cert\") pod \"5184d014-fc82-4576-8df6-85d71543c1d1\" (UID: \"5184d014-fc82-4576-8df6-85d71543c1d1\") " Jul 20 05:11:47.488726 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:47.488638 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwhqg\" (UniqueName: \"kubernetes.io/projected/5184d014-fc82-4576-8df6-85d71543c1d1-kube-api-access-kwhqg\") pod \"5184d014-fc82-4576-8df6-85d71543c1d1\" (UID: \"5184d014-fc82-4576-8df6-85d71543c1d1\") " Jul 20 05:11:47.490688 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:47.490663 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5184d014-fc82-4576-8df6-85d71543c1d1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5184d014-fc82-4576-8df6-85d71543c1d1" (UID: "5184d014-fc82-4576-8df6-85d71543c1d1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 05:11:47.490806 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:47.490773 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5184d014-fc82-4576-8df6-85d71543c1d1-kube-api-access-kwhqg" (OuterVolumeSpecName: "kube-api-access-kwhqg") pod "5184d014-fc82-4576-8df6-85d71543c1d1" (UID: "5184d014-fc82-4576-8df6-85d71543c1d1"). InnerVolumeSpecName "kube-api-access-kwhqg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:11:47.589889 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:47.589858 2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5184d014-fc82-4576-8df6-85d71543c1d1-webhook-cert\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:11:47.589889 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:47.589886 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwhqg\" (UniqueName: \"kubernetes.io/projected/5184d014-fc82-4576-8df6-85d71543c1d1-kube-api-access-kwhqg\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:11:48.122615 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:48.122577 2576 generic.go:356] "Generic (PLEG): container finished" podID="5184d014-fc82-4576-8df6-85d71543c1d1" containerID="307f38e90667b6f1b9c13a9fb9b16b1f4c21d7f0aa4f98f4d8d1bb4029ad0ff2" exitCode=0 Jul 20 05:11:48.123029 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:48.122619 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59db86fcdd-dmqjt" Jul 20 05:11:48.123029 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:48.122664 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59db86fcdd-dmqjt" event={"ID":"5184d014-fc82-4576-8df6-85d71543c1d1","Type":"ContainerDied","Data":"307f38e90667b6f1b9c13a9fb9b16b1f4c21d7f0aa4f98f4d8d1bb4029ad0ff2"} Jul 20 05:11:48.123029 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:48.122714 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59db86fcdd-dmqjt" event={"ID":"5184d014-fc82-4576-8df6-85d71543c1d1","Type":"ContainerDied","Data":"34de34526f550f3f6aa581e080c52ad09c5e72c4aec148b5168084b7d82ae801"} Jul 20 05:11:48.123029 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:48.122738 2576 scope.go:117] "RemoveContainer" containerID="307f38e90667b6f1b9c13a9fb9b16b1f4c21d7f0aa4f98f4d8d1bb4029ad0ff2" Jul 20 05:11:48.132179 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:48.132023 2576 scope.go:117] "RemoveContainer" containerID="307f38e90667b6f1b9c13a9fb9b16b1f4c21d7f0aa4f98f4d8d1bb4029ad0ff2" Jul 20 05:11:48.132298 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:11:48.132281 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"307f38e90667b6f1b9c13a9fb9b16b1f4c21d7f0aa4f98f4d8d1bb4029ad0ff2\": container with ID starting with 307f38e90667b6f1b9c13a9fb9b16b1f4c21d7f0aa4f98f4d8d1bb4029ad0ff2 not found: ID does not exist" containerID="307f38e90667b6f1b9c13a9fb9b16b1f4c21d7f0aa4f98f4d8d1bb4029ad0ff2" Jul 20 05:11:48.132338 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:48.132307 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"307f38e90667b6f1b9c13a9fb9b16b1f4c21d7f0aa4f98f4d8d1bb4029ad0ff2"} err="failed to get container status \"307f38e90667b6f1b9c13a9fb9b16b1f4c21d7f0aa4f98f4d8d1bb4029ad0ff2\": rpc error: code = NotFound desc = could not find container \"307f38e90667b6f1b9c13a9fb9b16b1f4c21d7f0aa4f98f4d8d1bb4029ad0ff2\": container with ID starting with 307f38e90667b6f1b9c13a9fb9b16b1f4c21d7f0aa4f98f4d8d1bb4029ad0ff2 not found: ID does not exist" Jul 20 05:11:48.138794 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:48.138772 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59db86fcdd-dmqjt"] Jul 20 05:11:48.142965 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:48.142937 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-59db86fcdd-dmqjt"] Jul 20 05:11:49.654101 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:11:49.654071 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5184d014-fc82-4576-8df6-85d71543c1d1" path="/var/lib/kubelet/pods/5184d014-fc82-4576-8df6-85d71543c1d1/volumes" Jul 20 05:12:07.829627 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:07.829583 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:12:07.934153 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:07.934114 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:12:08.482110 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:08.482075 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:12:08.977100 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:08.977021 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:12:09.069480 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.069443 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:12:09.073974 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.073937 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-p7lst"] Jul 20 05:12:09.074399 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.074385 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5184d014-fc82-4576-8df6-85d71543c1d1" containerName="manager" Jul 20 05:12:09.074453 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.074403 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5184d014-fc82-4576-8df6-85d71543c1d1" containerName="manager" Jul 20 05:12:09.074512 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.074501 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5184d014-fc82-4576-8df6-85d71543c1d1" containerName="manager" Jul 20 05:12:09.083224 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.083206 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-p7lst" Jul 20 05:12:09.085896 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.085869 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 20 05:12:09.086024 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.085872 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-68v75\"" Jul 20 05:12:09.088197 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.088177 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c58dn\" (UniqueName: \"kubernetes.io/projected/4c0b6951-8797-43f2-b26e-0d9b5ce04e71-kube-api-access-c58dn\") pod \"payload-pre-processing-7677b774bb-p7lst\" (UID: \"4c0b6951-8797-43f2-b26e-0d9b5ce04e71\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-p7lst" Jul 20 05:12:09.088443 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.088417 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c0b6951-8797-43f2-b26e-0d9b5ce04e71-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-p7lst\" (UID: \"4c0b6951-8797-43f2-b26e-0d9b5ce04e71\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-p7lst" Jul 20 05:12:09.090902 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.090878 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-p7lst"] Jul 20 05:12:09.113674 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.113650 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-fmsw5"] Jul 20 05:12:09.117619 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.117603 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-fmsw5" Jul 20 05:12:09.132734 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.132709 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-fmsw5"] Jul 20 05:12:09.189172 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.189132 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c58dn\" (UniqueName: \"kubernetes.io/projected/4c0b6951-8797-43f2-b26e-0d9b5ce04e71-kube-api-access-c58dn\") pod \"payload-pre-processing-7677b774bb-p7lst\" (UID: \"4c0b6951-8797-43f2-b26e-0d9b5ce04e71\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-p7lst" Jul 20 05:12:09.189344 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.189207 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aa67061-8af2-48f0-9c2c-ca47229be185-plugins-config-volume\") pod \"payload-processing-5bf47cd988-fmsw5\" (UID: \"0aa67061-8af2-48f0-9c2c-ca47229be185\") " pod="openshift-ingress/payload-processing-5bf47cd988-fmsw5" Jul 20 05:12:09.189344 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.189250 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c0b6951-8797-43f2-b26e-0d9b5ce04e71-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-p7lst\" (UID: \"4c0b6951-8797-43f2-b26e-0d9b5ce04e71\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-p7lst" Jul 20 05:12:09.189344 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.189324 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj5wl\" (UniqueName: \"kubernetes.io/projected/0aa67061-8af2-48f0-9c2c-ca47229be185-kube-api-access-mj5wl\") pod \"payload-processing-5bf47cd988-fmsw5\" (UID: \"0aa67061-8af2-48f0-9c2c-ca47229be185\") " pod="openshift-ingress/payload-processing-5bf47cd988-fmsw5" Jul 20 05:12:09.190438 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.190409 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c0b6951-8797-43f2-b26e-0d9b5ce04e71-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-p7lst\" (UID: \"4c0b6951-8797-43f2-b26e-0d9b5ce04e71\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-p7lst" Jul 20 05:12:09.209040 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.209006 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c58dn\" (UniqueName: \"kubernetes.io/projected/4c0b6951-8797-43f2-b26e-0d9b5ce04e71-kube-api-access-c58dn\") pod \"payload-pre-processing-7677b774bb-p7lst\" (UID: \"4c0b6951-8797-43f2-b26e-0d9b5ce04e71\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-p7lst" Jul 20 05:12:09.290858 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.290825 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj5wl\" (UniqueName: \"kubernetes.io/projected/0aa67061-8af2-48f0-9c2c-ca47229be185-kube-api-access-mj5wl\") pod \"payload-processing-5bf47cd988-fmsw5\" (UID: \"0aa67061-8af2-48f0-9c2c-ca47229be185\") " pod="openshift-ingress/payload-processing-5bf47cd988-fmsw5" Jul 20 05:12:09.291036 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.290977 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aa67061-8af2-48f0-9c2c-ca47229be185-plugins-config-volume\") pod \"payload-processing-5bf47cd988-fmsw5\" (UID: \"0aa67061-8af2-48f0-9c2c-ca47229be185\") " pod="openshift-ingress/payload-processing-5bf47cd988-fmsw5" Jul 20 05:12:09.291655 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.291631 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0aa67061-8af2-48f0-9c2c-ca47229be185-plugins-config-volume\") pod \"payload-processing-5bf47cd988-fmsw5\" (UID: \"0aa67061-8af2-48f0-9c2c-ca47229be185\") " pod="openshift-ingress/payload-processing-5bf47cd988-fmsw5" Jul 20 05:12:09.300829 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.300805 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj5wl\" (UniqueName: \"kubernetes.io/projected/0aa67061-8af2-48f0-9c2c-ca47229be185-kube-api-access-mj5wl\") pod \"payload-processing-5bf47cd988-fmsw5\" (UID: \"0aa67061-8af2-48f0-9c2c-ca47229be185\") " pod="openshift-ingress/payload-processing-5bf47cd988-fmsw5" Jul 20 05:12:09.335332 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.335302 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:12:09.396043 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.396015 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-p7lst" Jul 20 05:12:09.427808 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.427778 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-fmsw5" Jul 20 05:12:09.550217 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.550178 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-p7lst"] Jul 20 05:12:09.551001 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:12:09.550976 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c0b6951_8797_43f2_b26e_0d9b5ce04e71.slice/crio-c1f386e35e3e27b6903ee008817de367d148f636f098d5a6ce3adab9e162fc1b WatchSource:0}: Error finding container c1f386e35e3e27b6903ee008817de367d148f636f098d5a6ce3adab9e162fc1b: Status 404 returned error can't find the container with id c1f386e35e3e27b6903ee008817de367d148f636f098d5a6ce3adab9e162fc1b Jul 20 05:12:09.583779 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:09.583754 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-fmsw5"] Jul 20 05:12:09.584511 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:12:09.584486 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa67061_8af2_48f0_9c2c_ca47229be185.slice/crio-0012335149a85af1a9cda5e53263eebe68662a92a78147cf196211df5cffc30c WatchSource:0}: Error finding container 0012335149a85af1a9cda5e53263eebe68662a92a78147cf196211df5cffc30c: Status 404 returned error can't find the container with id 0012335149a85af1a9cda5e53263eebe68662a92a78147cf196211df5cffc30c Jul 20 05:12:10.265145 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:10.265104 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-fmsw5" event={"ID":"0aa67061-8af2-48f0-9c2c-ca47229be185","Type":"ContainerStarted","Data":"0012335149a85af1a9cda5e53263eebe68662a92a78147cf196211df5cffc30c"} Jul 20 05:12:10.267842 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:10.267799 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-p7lst" event={"ID":"4c0b6951-8797-43f2-b26e-0d9b5ce04e71","Type":"ContainerStarted","Data":"c1f386e35e3e27b6903ee008817de367d148f636f098d5a6ce3adab9e162fc1b"} Jul 20 05:12:12.285415 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:12.285374 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-p7lst" event={"ID":"4c0b6951-8797-43f2-b26e-0d9b5ce04e71","Type":"ContainerStarted","Data":"610dca309ee519accce649b66d864324ba6a94fab51ae30362bf7968e73161d5"} Jul 20 05:12:12.285843 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:12.285443 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-p7lst" Jul 20 05:12:12.286970 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:12.286935 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-fmsw5" event={"ID":"0aa67061-8af2-48f0-9c2c-ca47229be185","Type":"ContainerStarted","Data":"0335eb2ff805cb127aa805556b11972c945c95f0ef5914c420ec138ec83e0ac5"} Jul 20 05:12:12.287088 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:12.287001 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-fmsw5" Jul 20 05:12:12.299530 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:12.299483 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-p7lst" podStartSLOduration=1.134442794 podStartE2EDuration="3.299467189s" totalImagesPullingTime="2.165024395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:12:12.299441775 +0000 UTC m=+775.345968676" watchObservedRunningTime="2026-07-20 05:12:12.299467189 +0000 UTC m=+775.345994096" Jul 20 05:12:12.314003 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:12.313947 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-fmsw5" podStartSLOduration=1.175916427 podStartE2EDuration="3.313934238s" totalImagesPullingTime="2.138017811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:12:12.312575698 +0000 UTC m=+775.359102627" watchObservedRunningTime="2026-07-20 05:12:12.313934238 +0000 UTC m=+775.360461144" Jul 20 05:12:17.316828 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:17.316791 2576 generic.go:356] "Generic (PLEG): container finished" podID="88b632e5-2858-48b7-817a-5704fe6f3469" containerID="32b757c171c2c73ec412c9331af2725dd75a7f6b892fcc508d770e07906ba586" exitCode=0 Jul 20 05:12:17.317337 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:17.316837 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" event={"ID":"88b632e5-2858-48b7-817a-5704fe6f3469","Type":"ContainerDied","Data":"32b757c171c2c73ec412c9331af2725dd75a7f6b892fcc508d770e07906ba586"} Jul 20 05:12:17.317337 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:17.317199 2576 scope.go:117] "RemoveContainer" containerID="32b757c171c2c73ec412c9331af2725dd75a7f6b892fcc508d770e07906ba586" Jul 20 05:12:17.318167 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:17.318136 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:12:17.318245 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:17.318206 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:12:17.318245 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:17.318241 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:12:18.324779 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:18.324748 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" event={"ID":"88b632e5-2858-48b7-817a-5704fe6f3469","Type":"ContainerStarted","Data":"b89b015de096e5a2c8c64c0c85984577192d9a0084a91a123a7e382407644f9a"} Jul 20 05:12:18.889803 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:18.889764 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:12:18.890896 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:18.890866 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-r4zbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 20 05:12:18.891045 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:18.890917 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" podUID="88b632e5-2858-48b7-817a-5704fe6f3469" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 20 05:12:19.889849 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:19.889811 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-r4zbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 20 05:12:19.890284 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:19.889869 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" podUID="88b632e5-2858-48b7-817a-5704fe6f3469" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 20 05:12:20.559616 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:20.559565 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:12:20.672595 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:20.672565 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:12:20.890634 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:20.890549 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-r4zbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 20 05:12:20.890634 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:20.890607 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" podUID="88b632e5-2858-48b7-817a-5704fe6f3469" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 20 05:12:21.890480 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:21.890445 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-r4zbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 20 05:12:21.890670 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:21.890513 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" podUID="88b632e5-2858-48b7-817a-5704fe6f3469" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 20 05:12:22.700355 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:22.700322 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:12:22.894470 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:22.894439 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:12:23.295124 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:23.295093 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-fmsw5" Jul 20 05:12:23.295292 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:23.295157 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-p7lst" Jul 20 05:12:23.360487 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:23.360459 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-r4zbh" Jul 20 05:12:31.414233 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.414204 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m"] Jul 20 05:12:31.419180 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.419162 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.421460 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.421439 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 20 05:12:31.422563 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.422543 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pg8cj\"" Jul 20 05:12:31.422661 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.422635 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 20 05:12:31.422798 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.422782 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 20 05:12:31.425546 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.425522 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m"] Jul 20 05:12:31.441200 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.441174 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:12:31.507936 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.507905 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vdfs\" (UniqueName: \"kubernetes.io/projected/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-kube-api-access-2vdfs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.508098 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.507939 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.508098 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.508072 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.508201 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.508130 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.508201 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.508173 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.508201 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.508195 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.608705 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.608669 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.608883 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.608723 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.608883 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.608767 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.608883 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.608788 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.609078 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.608898 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vdfs\" (UniqueName: \"kubernetes.io/projected/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-kube-api-access-2vdfs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.609078 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.608930 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.609168 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.609141 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.609227 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.609197 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.609263 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.609229 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.611241 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.611209 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.611482 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.611465 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.616156 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.616133 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vdfs\" (UniqueName: \"kubernetes.io/projected/4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199-kube-api-access-2vdfs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m\" (UID: \"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.731653 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.731629 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:31.864670 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:31.864524 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m"] Jul 20 05:12:31.867631 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:12:31.867603 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e3b621d_6ba5_4d6b_a91d_6d8f9c01b199.slice/crio-1edbb901d3709e0bb2ab48e81b0ed2a2ea9317d5149eaaffdc6193cd2dc61398 WatchSource:0}: Error finding container 1edbb901d3709e0bb2ab48e81b0ed2a2ea9317d5149eaaffdc6193cd2dc61398: Status 404 returned error can't find the container with id 1edbb901d3709e0bb2ab48e81b0ed2a2ea9317d5149eaaffdc6193cd2dc61398 Jul 20 05:12:32.417874 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:32.417796 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" event={"ID":"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199","Type":"ContainerStarted","Data":"1edbb901d3709e0bb2ab48e81b0ed2a2ea9317d5149eaaffdc6193cd2dc61398"} Jul 20 05:12:33.424926 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:33.424834 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" event={"ID":"4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199","Type":"ContainerStarted","Data":"364aac03ceebb9556966b9cfddb8d92f3c89b5c4c2c0fa6aef589d6b434dcb3e"} Jul 20 05:12:33.441522 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:33.441471 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" podStartSLOduration=1.170927007 podStartE2EDuration="2.44145762s" totalImagesPullingTime="1.270530613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:12:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:12:33.438880479 +0000 UTC m=+796.485407384" watchObservedRunningTime="2026-07-20 05:12:33.44145762 +0000 UTC m=+796.487984526" Jul 20 05:12:41.732409 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:41.732326 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:41.732409 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:41.732394 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:41.744534 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:41.744509 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:42.489336 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:42.489308 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m" Jul 20 05:12:46.853396 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:46.853364 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:12:48.650464 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:48.650430 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:12:59.144916 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:12:59.144884 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:13:05.622456 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.622420 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h"] Jul 20 05:13:05.627206 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.627187 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.629974 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.629931 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 20 05:13:05.631995 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.631967 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h"] Jul 20 05:13:05.643162 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.643138 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:13:05.718643 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.718605 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.718798 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.718649 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.718798 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.718668 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8w5n\" (UniqueName: \"kubernetes.io/projected/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-kube-api-access-q8w5n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.718798 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.718718 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.718798 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.718743 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.719026 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.718898 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.819569 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.819534 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.819569 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.819573 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.819812 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.819600 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.819812 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.819621 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8w5n\" (UniqueName: \"kubernetes.io/projected/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-kube-api-access-q8w5n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.819812 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.819659 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.819812 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.819684 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.820053 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.819923 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.820053 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.820022 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.820145 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.820069 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.822071 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.822046 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.822263 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.822248 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.826782 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.826762 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8w5n\" (UniqueName: \"kubernetes.io/projected/92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5-kube-api-access-q8w5n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h\" (UID: \"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:05.938564 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:05.938487 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:06.076293 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:06.076261 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h"] Jul 20 05:13:06.076753 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:13:06.076722 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e8e2cd_3a6e_4fc8_b981_d8c0a11a32d5.slice/crio-ec79155c60ee2987dcd25e3646a6fbe718973dd5791ba2490bdc1f1b34bd45af WatchSource:0}: Error finding container ec79155c60ee2987dcd25e3646a6fbe718973dd5791ba2490bdc1f1b34bd45af: Status 404 returned error can't find the container with id ec79155c60ee2987dcd25e3646a6fbe718973dd5791ba2490bdc1f1b34bd45af Jul 20 05:13:06.610342 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:06.610254 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" event={"ID":"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5","Type":"ContainerStarted","Data":"28b04fb45582f647b9a5947272c92bed52802a19e84e4f0bee3741dc41235b09"} Jul 20 05:13:06.610342 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:06.610320 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" event={"ID":"92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5","Type":"ContainerStarted","Data":"ec79155c60ee2987dcd25e3646a6fbe718973dd5791ba2490bdc1f1b34bd45af"} Jul 20 05:13:06.628214 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:06.628168 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" podStartSLOduration=1.444693767 podStartE2EDuration="1.628152367s" totalImagesPullingTime="183.4586ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:13:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:13:06.625349428 +0000 UTC m=+829.671876359" watchObservedRunningTime="2026-07-20 05:13:06.628152367 +0000 UTC m=+829.674679273" Jul 20 05:13:15.939638 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:15.939602 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:15.940095 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:15.939683 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:15.952473 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:15.952447 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:16.683629 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:16.683590 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h" Jul 20 05:13:32.024124 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:32.024091 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:13:50.558330 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:13:50.558291 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:14:17.711625 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:14:17.711597 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:14:17.712809 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:14:17.712788 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:14:17.716728 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:14:17.716707 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:14:17.718501 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:14:17.718486 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:14:27.752719 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:14:27.752685 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:15:00.141708 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:00.141672 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742075-wxcvr"] Jul 20 05:15:00.145611 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:00.145593 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742075-wxcvr" Jul 20 05:15:00.148362 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:00.148336 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:15:00.150579 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:00.150551 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742075-wxcvr"] Jul 20 05:15:00.158741 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:00.158715 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742075-wxcvr" Jul 20 05:15:00.503596 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:00.503571 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742075-wxcvr"] Jul 20 05:15:00.505801 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:15:00.505776 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62cc1edf_1bf4_4310_ace5_86f79902a2cb.slice/crio-e2cc111bb1872150e94427624f63460dc7c12c053355b1c20a0090e41e7821d8 WatchSource:0}: Error finding container e2cc111bb1872150e94427624f63460dc7c12c053355b1c20a0090e41e7821d8: Status 404 returned error can't find the container with id e2cc111bb1872150e94427624f63460dc7c12c053355b1c20a0090e41e7821d8 Jul 20 05:15:00.507675 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:00.507657 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 05:15:01.286670 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:01.286628 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742075-wxcvr" event={"ID":"62cc1edf-1bf4-4310-ace5-86f79902a2cb","Type":"ContainerStarted","Data":"e2cc111bb1872150e94427624f63460dc7c12c053355b1c20a0090e41e7821d8"} Jul 20 05:15:03.300211 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:03.300172 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742075-wxcvr" event={"ID":"62cc1edf-1bf4-4310-ace5-86f79902a2cb","Type":"ContainerStarted","Data":"2b7431c779b5476d1658e1c65cbff799b0702f390be90daf3a85367192297fe7"} Jul 20 05:15:03.316742 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:03.316692 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742075-wxcvr" podStartSLOduration=1.300099806 podStartE2EDuration="3.31667743s" totalImagesPullingTime="2.016577624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:15:03.314840997 +0000 UTC m=+946.361367929" watchObservedRunningTime="2026-07-20 05:15:03.31667743 +0000 UTC m=+946.363204335" Jul 20 05:15:04.306931 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:04.306894 2576 generic.go:356] "Generic (PLEG): container finished" podID="62cc1edf-1bf4-4310-ace5-86f79902a2cb" containerID="2b7431c779b5476d1658e1c65cbff799b0702f390be90daf3a85367192297fe7" exitCode=0 Jul 20 05:15:04.307346 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:04.306984 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742075-wxcvr" event={"ID":"62cc1edf-1bf4-4310-ace5-86f79902a2cb","Type":"ContainerDied","Data":"2b7431c779b5476d1658e1c65cbff799b0702f390be90daf3a85367192297fe7"} Jul 20 05:15:05.451629 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:05.451603 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742075-wxcvr" Jul 20 05:15:06.320538 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:06.320504 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742075-wxcvr" event={"ID":"62cc1edf-1bf4-4310-ace5-86f79902a2cb","Type":"ContainerDied","Data":"e2cc111bb1872150e94427624f63460dc7c12c053355b1c20a0090e41e7821d8"} Jul 20 05:15:06.320538 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:06.320537 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742075-wxcvr" Jul 20 05:15:06.320792 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:06.320553 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2cc111bb1872150e94427624f63460dc7c12c053355b1c20a0090e41e7821d8" Jul 20 05:15:07.860010 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:07.859929 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:15:07.961299 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:07.961263 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:15:09.010855 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:09.010820 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:15:09.113685 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:09.113652 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:15:56.580305 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:56.580222 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-n2qnp"] Jul 20 05:15:56.580838 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:56.580432 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" podUID="5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4" containerName="manager" containerID="cri-o://56102ca47fd499b8e1cd2510afad5588f61f26024533e3c20750617ce7c03a89" gracePeriod=30 Jul 20 05:15:56.838129 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:56.838067 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" Jul 20 05:15:56.948332 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:56.948295 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rmxv\" (UniqueName: \"kubernetes.io/projected/5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4-kube-api-access-4rmxv\") pod \"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4\" (UID: \"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4\") " Jul 20 05:15:56.948534 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:56.948377 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4-webhook-cert\") pod \"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4\" (UID: \"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4\") " Jul 20 05:15:56.950464 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:56.950435 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4" (UID: "5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 05:15:56.950579 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:56.950526 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4-kube-api-access-4rmxv" (OuterVolumeSpecName: "kube-api-access-4rmxv") pod "5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4" (UID: "5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4"). InnerVolumeSpecName "kube-api-access-4rmxv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:15:57.050092 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:57.050055 2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4-webhook-cert\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:15:57.050092 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:57.050085 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rmxv\" (UniqueName: \"kubernetes.io/projected/5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4-kube-api-access-4rmxv\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:15:57.621708 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:57.621674 2576 generic.go:356] "Generic (PLEG): container finished" podID="5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4" containerID="56102ca47fd499b8e1cd2510afad5588f61f26024533e3c20750617ce7c03a89" exitCode=0 Jul 20 05:15:57.622150 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:57.621719 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" Jul 20 05:15:57.622150 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:57.621758 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" event={"ID":"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4","Type":"ContainerDied","Data":"56102ca47fd499b8e1cd2510afad5588f61f26024533e3c20750617ce7c03a89"} Jul 20 05:15:57.622150 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:57.621817 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4f66bf88-n2qnp" event={"ID":"5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4","Type":"ContainerDied","Data":"0788e1689cd0b2d76d9a2b2fce1a656306c3d755a262e87e4362eb09095a6dd1"} Jul 20 05:15:57.622150 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:57.621845 2576 scope.go:117] "RemoveContainer" containerID="56102ca47fd499b8e1cd2510afad5588f61f26024533e3c20750617ce7c03a89" Jul 20 05:15:57.631874 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:57.631855 2576 scope.go:117] "RemoveContainer" containerID="56102ca47fd499b8e1cd2510afad5588f61f26024533e3c20750617ce7c03a89" Jul 20 05:15:57.632157 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:15:57.632135 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56102ca47fd499b8e1cd2510afad5588f61f26024533e3c20750617ce7c03a89\": container with ID starting with 56102ca47fd499b8e1cd2510afad5588f61f26024533e3c20750617ce7c03a89 not found: ID does not exist" containerID="56102ca47fd499b8e1cd2510afad5588f61f26024533e3c20750617ce7c03a89" Jul 20 05:15:57.632234 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:57.632171 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56102ca47fd499b8e1cd2510afad5588f61f26024533e3c20750617ce7c03a89"} err="failed to get container status \"56102ca47fd499b8e1cd2510afad5588f61f26024533e3c20750617ce7c03a89\": rpc error: code = NotFound desc = could not find container \"56102ca47fd499b8e1cd2510afad5588f61f26024533e3c20750617ce7c03a89\": container with ID starting with 56102ca47fd499b8e1cd2510afad5588f61f26024533e3c20750617ce7c03a89 not found: ID does not exist" Jul 20 05:15:57.643837 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:57.643808 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-n2qnp"] Jul 20 05:15:57.647028 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:57.647001 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-n2qnp"] Jul 20 05:15:57.653937 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:15:57.653914 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4" path="/var/lib/kubelet/pods/5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4/volumes" Jul 20 05:16:56.057852 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:56.057818 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:16:56.147727 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:56.147689 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:16:56.622093 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:56.622063 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:16:56.726447 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:56.726411 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:16:58.486790 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.486751 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-z8wmg"] Jul 20 05:16:58.487213 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.487195 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62cc1edf-1bf4-4310-ace5-86f79902a2cb" containerName="cleanup" Jul 20 05:16:58.487213 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.487207 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="62cc1edf-1bf4-4310-ace5-86f79902a2cb" containerName="cleanup" Jul 20 05:16:58.487298 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.487231 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4" containerName="manager" Jul 20 05:16:58.487298 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.487237 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4" containerName="manager" Jul 20 05:16:58.487373 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.487310 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb7e7a4-9e49-4a5d-a08b-d0620e5753a4" containerName="manager" Jul 20 05:16:58.487373 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.487320 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="62cc1edf-1bf4-4310-ace5-86f79902a2cb" containerName="cleanup" Jul 20 05:16:58.489938 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.489921 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" Jul 20 05:16:58.492412 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.492387 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lqrrn\"" Jul 20 05:16:58.492412 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.492404 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 20 05:16:58.492588 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.492422 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 20 05:16:58.500263 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.500238 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-z8wmg"] Jul 20 05:16:58.544215 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.544174 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cxpl\" (UniqueName: \"kubernetes.io/projected/a784a18b-959b-4253-9275-6da1a68e6259-kube-api-access-6cxpl\") pod \"maas-controller-6b4f66bf88-z8wmg\" (UID: \"a784a18b-959b-4253-9275-6da1a68e6259\") " pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" Jul 20 05:16:58.544415 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.544225 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a784a18b-959b-4253-9275-6da1a68e6259-webhook-cert\") pod \"maas-controller-6b4f66bf88-z8wmg\" (UID: \"a784a18b-959b-4253-9275-6da1a68e6259\") " pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" Jul 20 05:16:58.645181 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.645125 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cxpl\" (UniqueName: \"kubernetes.io/projected/a784a18b-959b-4253-9275-6da1a68e6259-kube-api-access-6cxpl\") pod \"maas-controller-6b4f66bf88-z8wmg\" (UID: \"a784a18b-959b-4253-9275-6da1a68e6259\") " pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" Jul 20 05:16:58.645181 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.645192 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a784a18b-959b-4253-9275-6da1a68e6259-webhook-cert\") pod \"maas-controller-6b4f66bf88-z8wmg\" (UID: \"a784a18b-959b-4253-9275-6da1a68e6259\") " pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" Jul 20 05:16:58.647602 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.647578 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a784a18b-959b-4253-9275-6da1a68e6259-webhook-cert\") pod \"maas-controller-6b4f66bf88-z8wmg\" (UID: \"a784a18b-959b-4253-9275-6da1a68e6259\") " pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" Jul 20 05:16:58.652540 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.652519 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cxpl\" (UniqueName: \"kubernetes.io/projected/a784a18b-959b-4253-9275-6da1a68e6259-kube-api-access-6cxpl\") pod \"maas-controller-6b4f66bf88-z8wmg\" (UID: \"a784a18b-959b-4253-9275-6da1a68e6259\") " pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" Jul 20 05:16:58.802420 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:58.802323 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" Jul 20 05:16:59.145582 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:59.145551 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-z8wmg"] Jul 20 05:16:59.146488 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:16:59.146462 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda784a18b_959b_4253_9275_6da1a68e6259.slice/crio-62d5a1d0eb7794620e4e738ed802b7faef56c3b77df7c7839671ec89025e39ba WatchSource:0}: Error finding container 62d5a1d0eb7794620e4e738ed802b7faef56c3b77df7c7839671ec89025e39ba: Status 404 returned error can't find the container with id 62d5a1d0eb7794620e4e738ed802b7faef56c3b77df7c7839671ec89025e39ba Jul 20 05:16:59.980163 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:59.980131 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" event={"ID":"a784a18b-959b-4253-9275-6da1a68e6259","Type":"ContainerStarted","Data":"32e40a86a7d98428acf9f0245ee27ceb545d423c2477e0148f4b0b43660234b1"} Jul 20 05:16:59.980547 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:59.980192 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" event={"ID":"a784a18b-959b-4253-9275-6da1a68e6259","Type":"ContainerStarted","Data":"62d5a1d0eb7794620e4e738ed802b7faef56c3b77df7c7839671ec89025e39ba"} Jul 20 05:16:59.980547 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:59.980316 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" Jul 20 05:16:59.998088 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:16:59.998025 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" podStartSLOduration=1.369501546 podStartE2EDuration="1.998003539s" totalImagesPullingTime="628.501993ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:16:59.997543641 +0000 UTC m=+1063.044070562" watchObservedRunningTime="2026-07-20 05:16:59.998003539 +0000 UTC m=+1063.044530447" Jul 20 05:17:10.988928 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:10.988847 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" Jul 20 05:17:15.517809 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:15.517766 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:17:15.629672 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:15.629634 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-z8wmg"] Jul 20 05:17:15.629895 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:15.629854 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" podUID="a784a18b-959b-4253-9275-6da1a68e6259" containerName="manager" containerID="cri-o://32e40a86a7d98428acf9f0245ee27ceb545d423c2477e0148f4b0b43660234b1" gracePeriod=30 Jul 20 05:17:15.891706 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:15.891677 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" Jul 20 05:17:16.015266 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.015234 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a784a18b-959b-4253-9275-6da1a68e6259-webhook-cert\") pod \"a784a18b-959b-4253-9275-6da1a68e6259\" (UID: \"a784a18b-959b-4253-9275-6da1a68e6259\") " Jul 20 05:17:16.015415 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.015283 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cxpl\" (UniqueName: \"kubernetes.io/projected/a784a18b-959b-4253-9275-6da1a68e6259-kube-api-access-6cxpl\") pod \"a784a18b-959b-4253-9275-6da1a68e6259\" (UID: \"a784a18b-959b-4253-9275-6da1a68e6259\") " Jul 20 05:17:16.017385 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.017351 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a784a18b-959b-4253-9275-6da1a68e6259-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a784a18b-959b-4253-9275-6da1a68e6259" (UID: "a784a18b-959b-4253-9275-6da1a68e6259"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 05:17:16.017484 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.017385 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a784a18b-959b-4253-9275-6da1a68e6259-kube-api-access-6cxpl" (OuterVolumeSpecName: "kube-api-access-6cxpl") pod "a784a18b-959b-4253-9275-6da1a68e6259" (UID: "a784a18b-959b-4253-9275-6da1a68e6259"). InnerVolumeSpecName "kube-api-access-6cxpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:17:16.071196 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.071111 2576 generic.go:356] "Generic (PLEG): container finished" podID="a784a18b-959b-4253-9275-6da1a68e6259" containerID="32e40a86a7d98428acf9f0245ee27ceb545d423c2477e0148f4b0b43660234b1" exitCode=0 Jul 20 05:17:16.071196 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.071154 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" event={"ID":"a784a18b-959b-4253-9275-6da1a68e6259","Type":"ContainerDied","Data":"32e40a86a7d98428acf9f0245ee27ceb545d423c2477e0148f4b0b43660234b1"} Jul 20 05:17:16.071196 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.071158 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" Jul 20 05:17:16.071196 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.071198 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4f66bf88-z8wmg" event={"ID":"a784a18b-959b-4253-9275-6da1a68e6259","Type":"ContainerDied","Data":"62d5a1d0eb7794620e4e738ed802b7faef56c3b77df7c7839671ec89025e39ba"} Jul 20 05:17:16.071498 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.071219 2576 scope.go:117] "RemoveContainer" containerID="32e40a86a7d98428acf9f0245ee27ceb545d423c2477e0148f4b0b43660234b1" Jul 20 05:17:16.080683 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.080657 2576 scope.go:117] "RemoveContainer" containerID="32e40a86a7d98428acf9f0245ee27ceb545d423c2477e0148f4b0b43660234b1" Jul 20 05:17:16.080978 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:17:16.080931 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32e40a86a7d98428acf9f0245ee27ceb545d423c2477e0148f4b0b43660234b1\": container with ID starting with 32e40a86a7d98428acf9f0245ee27ceb545d423c2477e0148f4b0b43660234b1 not found: ID does not exist" containerID="32e40a86a7d98428acf9f0245ee27ceb545d423c2477e0148f4b0b43660234b1" Jul 20 05:17:16.081051 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.080979 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32e40a86a7d98428acf9f0245ee27ceb545d423c2477e0148f4b0b43660234b1"} err="failed to get container status \"32e40a86a7d98428acf9f0245ee27ceb545d423c2477e0148f4b0b43660234b1\": rpc error: code = NotFound desc = could not find container \"32e40a86a7d98428acf9f0245ee27ceb545d423c2477e0148f4b0b43660234b1\": container with ID starting with 32e40a86a7d98428acf9f0245ee27ceb545d423c2477e0148f4b0b43660234b1 not found: ID does not exist" Jul 20 05:17:16.091767 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.091718 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-z8wmg"] Jul 20 05:17:16.094502 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.094480 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-z8wmg"] Jul 20 05:17:16.116297 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.116274 2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a784a18b-959b-4253-9275-6da1a68e6259-webhook-cert\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:17:16.116297 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.116297 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cxpl\" (UniqueName: \"kubernetes.io/projected/a784a18b-959b-4253-9275-6da1a68e6259-kube-api-access-6cxpl\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:17:16.808301 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.808269 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:17:16.910699 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.910664 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-dmxd7"] Jul 20 05:17:16.911121 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.911108 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a784a18b-959b-4253-9275-6da1a68e6259" containerName="manager" Jul 20 05:17:16.911181 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.911122 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a784a18b-959b-4253-9275-6da1a68e6259" containerName="manager" Jul 20 05:17:16.911217 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.911193 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a784a18b-959b-4253-9275-6da1a68e6259" containerName="manager" Jul 20 05:17:16.915497 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.915474 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4f66bf88-dmxd7" Jul 20 05:17:16.918928 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.918904 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lqrrn\"" Jul 20 05:17:16.919081 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.918972 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 20 05:17:16.919081 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.918950 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 20 05:17:16.922175 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:16.922152 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-dmxd7"] Jul 20 05:17:17.026379 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:17.026337 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36285f66-63b4-4188-befb-88219bed772f-webhook-cert\") pod \"maas-controller-6b4f66bf88-dmxd7\" (UID: \"36285f66-63b4-4188-befb-88219bed772f\") " pod="opendatahub/maas-controller-6b4f66bf88-dmxd7" Jul 20 05:17:17.026568 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:17.026513 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kprkh\" (UniqueName: \"kubernetes.io/projected/36285f66-63b4-4188-befb-88219bed772f-kube-api-access-kprkh\") pod \"maas-controller-6b4f66bf88-dmxd7\" (UID: \"36285f66-63b4-4188-befb-88219bed772f\") " pod="opendatahub/maas-controller-6b4f66bf88-dmxd7" Jul 20 05:17:17.128012 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:17.127898 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kprkh\" (UniqueName: \"kubernetes.io/projected/36285f66-63b4-4188-befb-88219bed772f-kube-api-access-kprkh\") pod \"maas-controller-6b4f66bf88-dmxd7\" (UID: \"36285f66-63b4-4188-befb-88219bed772f\") " pod="opendatahub/maas-controller-6b4f66bf88-dmxd7" Jul 20 05:17:17.128012 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:17.127946 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36285f66-63b4-4188-befb-88219bed772f-webhook-cert\") pod \"maas-controller-6b4f66bf88-dmxd7\" (UID: \"36285f66-63b4-4188-befb-88219bed772f\") " pod="opendatahub/maas-controller-6b4f66bf88-dmxd7" Jul 20 05:17:17.130434 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:17.130405 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36285f66-63b4-4188-befb-88219bed772f-webhook-cert\") pod \"maas-controller-6b4f66bf88-dmxd7\" (UID: \"36285f66-63b4-4188-befb-88219bed772f\") " pod="opendatahub/maas-controller-6b4f66bf88-dmxd7" Jul 20 05:17:17.136019 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:17.135998 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kprkh\" (UniqueName: \"kubernetes.io/projected/36285f66-63b4-4188-befb-88219bed772f-kube-api-access-kprkh\") pod \"maas-controller-6b4f66bf88-dmxd7\" (UID: \"36285f66-63b4-4188-befb-88219bed772f\") " pod="opendatahub/maas-controller-6b4f66bf88-dmxd7" Jul 20 05:17:17.228842 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:17.228804 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4f66bf88-dmxd7" Jul 20 05:17:17.575265 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:17.575241 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b4f66bf88-dmxd7"] Jul 20 05:17:17.578055 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:17:17.578025 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36285f66_63b4_4188_befb_88219bed772f.slice/crio-528a72eac9ee1a81b07af5c1336de580132f4acc3d0e87f891dd2b17d1db4bdd WatchSource:0}: Error finding container 528a72eac9ee1a81b07af5c1336de580132f4acc3d0e87f891dd2b17d1db4bdd: Status 404 returned error can't find the container with id 528a72eac9ee1a81b07af5c1336de580132f4acc3d0e87f891dd2b17d1db4bdd Jul 20 05:17:17.660764 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:17.660731 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a784a18b-959b-4253-9275-6da1a68e6259" path="/var/lib/kubelet/pods/a784a18b-959b-4253-9275-6da1a68e6259/volumes" Jul 20 05:17:18.085016 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:18.084982 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4f66bf88-dmxd7" event={"ID":"36285f66-63b4-4188-befb-88219bed772f","Type":"ContainerStarted","Data":"528a72eac9ee1a81b07af5c1336de580132f4acc3d0e87f891dd2b17d1db4bdd"} Jul 20 05:17:18.117315 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:18.117293 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 20 05:17:19.092356 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:19.092260 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4f66bf88-dmxd7" event={"ID":"36285f66-63b4-4188-befb-88219bed772f","Type":"ContainerStarted","Data":"e81eb2421a447d2dda493291ec93ec594fe604b2f31d098d253617d3d71f8fef"} Jul 20 05:17:19.092356 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:19.092331 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b4f66bf88-dmxd7" Jul 20 05:17:19.109971 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:19.109905 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b4f66bf88-dmxd7" podStartSLOduration=2.5747145849999997 podStartE2EDuration="3.109890248s" totalImagesPullingTime="535.175663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:17:19.107182138 +0000 UTC m=+1082.153709065" watchObservedRunningTime="2026-07-20 05:17:19.109890248 +0000 UTC m=+1082.156417180" Jul 20 05:17:30.101071 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:30.101038 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b4f66bf88-dmxd7" Jul 20 05:17:51.750185 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:51.750146 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:17:51.848541 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:51.848500 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:17:52.059097 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:52.058996 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:17:57.546021 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:57.545943 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:17:57.654118 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:57.654085 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:17:57.746693 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:57.746658 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:17:57.946696 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:17:57.946605 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:18:03.694292 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:18:03.694253 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:18:03.794079 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:18:03.794036 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:18:17.008447 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:18:17.008412 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:18:26.093689 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:18:26.093635 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:19:17.753335 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:19:17.753299 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:19:17.753817 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:19:17.753500 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:19:17.760839 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:19:17.760798 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:19:17.761079 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:19:17.761062 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:20:34.950509 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:20:34.950432 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:24:17.795617 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:24:17.795587 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:24:17.796613 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:24:17.796594 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:24:17.799838 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:24:17.799819 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:24:17.800783 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:24:17.800761 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:25:39.422814 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:25:39.422778 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:26:02.624269 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:26:02.624238 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:26:17.315769 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:26:17.315735 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:26:57.358347 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:26:57.358310 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:26:58.343724 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:26:58.343688 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:28:04.145706 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:28:04.145665 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:28:04.284332 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:28:04.284297 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:28:08.133297 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:28:08.133264 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:29:17.846857 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:29:17.846829 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:29:17.848918 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:29:17.848894 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:29:17.851008 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:29:17.850990 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:29:17.852981 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:29:17.852948 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:30:00.135212 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:00.135174 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742090-dwrvh"] Jul 20 05:30:00.139013 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:00.138996 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742090-dwrvh" Jul 20 05:30:00.141323 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:00.141305 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:30:00.148326 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:00.148307 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742090-dwrvh" Jul 20 05:30:00.155391 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:00.155370 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742090-dwrvh"] Jul 20 05:30:00.283214 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:00.283188 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742090-dwrvh"] Jul 20 05:30:00.284377 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:30:00.284352 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd84a4ee6_0f61_4d57_b17e_3b47d8d2c477.slice/crio-4761ab8fbd02da68c615dde7d19f6b2e2725baf37c14c81631273c0b495f457d WatchSource:0}: Error finding container 4761ab8fbd02da68c615dde7d19f6b2e2725baf37c14c81631273c0b495f457d: Status 404 returned error can't find the container with id 4761ab8fbd02da68c615dde7d19f6b2e2725baf37c14c81631273c0b495f457d Jul 20 05:30:00.443297 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:00.443269 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742090-dwrvh" event={"ID":"d84a4ee6-0f61-4d57-b17e-3b47d8d2c477","Type":"ContainerStarted","Data":"4761ab8fbd02da68c615dde7d19f6b2e2725baf37c14c81631273c0b495f457d"} Jul 20 05:30:01.450433 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:01.450349 2576 generic.go:356] "Generic (PLEG): container finished" podID="d84a4ee6-0f61-4d57-b17e-3b47d8d2c477" containerID="65d50080511a4026a227a2c7b96f8a5962acbf3efa8591c33cf33c643141d580" exitCode=0 Jul 20 05:30:01.450433 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:01.450386 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742090-dwrvh" event={"ID":"d84a4ee6-0f61-4d57-b17e-3b47d8d2c477","Type":"ContainerDied","Data":"65d50080511a4026a227a2c7b96f8a5962acbf3efa8591c33cf33c643141d580"} Jul 20 05:30:02.595659 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:02.595636 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742090-dwrvh" Jul 20 05:30:03.462634 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:03.462608 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742090-dwrvh" Jul 20 05:30:03.462802 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:03.462637 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742090-dwrvh" event={"ID":"d84a4ee6-0f61-4d57-b17e-3b47d8d2c477","Type":"ContainerDied","Data":"4761ab8fbd02da68c615dde7d19f6b2e2725baf37c14c81631273c0b495f457d"} Jul 20 05:30:03.462802 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:03.462677 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4761ab8fbd02da68c615dde7d19f6b2e2725baf37c14c81631273c0b495f457d" Jul 20 05:30:07.060269 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:07.060233 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:18.076780 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:18.076745 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:26.314268 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:26.314233 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:31.549431 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:31.549385 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:35.248237 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:35.248168 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:35.348309 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:35.348279 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:35.777637 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:35.777592 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:41.362536 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:41.362418 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:42.077344 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.077309 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h"] Jul 20 05:30:42.077978 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.077931 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d84a4ee6-0f61-4d57-b17e-3b47d8d2c477" containerName="cleanup" Jul 20 05:30:42.077978 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.077972 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d84a4ee6-0f61-4d57-b17e-3b47d8d2c477" containerName="cleanup" Jul 20 05:30:42.078186 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.078122 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d84a4ee6-0f61-4d57-b17e-3b47d8d2c477" containerName="cleanup" Jul 20 05:30:42.081806 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.081786 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.084273 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.084253 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-50b57cd2-openshift-default-dockercfg-rczz7\"" Jul 20 05:30:42.092344 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.092317 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h"] Jul 20 05:30:42.113656 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.113621 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-podinfo\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.113774 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.113666 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istiod-ca-cert\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.113774 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.113695 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-envoy\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.113774 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.113757 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-workload-socket\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.113935 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.113802 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-credential-socket\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.113935 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.113832 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-token\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.113935 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.113865 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-data\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.113935 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.113903 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-workload-certs\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.114114 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.114022 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsrww\" (UniqueName: \"kubernetes.io/projected/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-kube-api-access-wsrww\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.215169 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.215136 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-workload-socket\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.215358 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.215178 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-credential-socket\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.215358 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.215214 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-token\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.215358 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.215250 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-data\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.215358 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.215281 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-workload-certs\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.215575 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.215383 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsrww\" (UniqueName: \"kubernetes.io/projected/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-kube-api-access-wsrww\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.215575 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.215425 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-podinfo\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.215575 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.215464 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istiod-ca-cert\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.215575 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.215498 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-envoy\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.215832 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.215594 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-workload-socket\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.215832 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.215608 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-credential-socket\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.215832 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.215692 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-data\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.215832 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.215741 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-workload-certs\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.216221 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.216199 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istiod-ca-cert\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.217790 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.217766 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-envoy\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.218106 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.218086 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-podinfo\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.223799 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.223773 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsrww\" (UniqueName: \"kubernetes.io/projected/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-kube-api-access-wsrww\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.225034 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.225014 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-token\") pod \"e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.392008 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.391886 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:42.394917 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.394888 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:42.537346 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.537318 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h"] Jul 20 05:30:42.541551 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:30:42.541514 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7c270fc_30ef_4091_ad7c_d5d53ab5a832.slice/crio-993ae3bee29e010c0c1b4ea7d44781646de7954f28291817d4c81774107951ad WatchSource:0}: Error finding container 993ae3bee29e010c0c1b4ea7d44781646de7954f28291817d4c81774107951ad: Status 404 returned error can't find the container with id 993ae3bee29e010c0c1b4ea7d44781646de7954f28291817d4c81774107951ad Jul 20 05:30:42.543671 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.543637 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:30:42.543771 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.543702 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:30:42.543771 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.543731 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:30:42.686735 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.686654 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" event={"ID":"b7c270fc-30ef-4091-ad7c-d5d53ab5a832","Type":"ContainerStarted","Data":"031415adbcd35b92bffadda498bd134cfe92817805713d1d884e10393377f836"} Jul 20 05:30:42.686735 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.686711 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" event={"ID":"b7c270fc-30ef-4091-ad7c-d5d53ab5a832","Type":"ContainerStarted","Data":"993ae3bee29e010c0c1b4ea7d44781646de7954f28291817d4c81774107951ad"} Jul 20 05:30:42.709287 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:42.709234 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" podStartSLOduration=0.709218473 podStartE2EDuration="709.218473ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:30:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:30:42.707020113 +0000 UTC m=+1885.753547045" watchObservedRunningTime="2026-07-20 05:30:42.709218473 +0000 UTC m=+1885.755745382" Jul 20 05:30:43.395105 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:43.395071 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:43.400068 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:43.400042 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:43.692497 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:43.692421 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:43.693382 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:43.693364 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:30:44.032275 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:44.032240 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:44.109660 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:44.109623 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:44.176679 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:44.176648 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:44.348040 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:44.347942 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:44.699982 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:44.699870 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:44.804035 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:44.804006 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:30:59.266164 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:59.265289 2576 patch_prober.go:28] interesting pod/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": read tcp 10.133.0.2:52020->10.133.0.59:15021: read: connection reset by peer" start-of-body= Jul 20 05:30:59.266164 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:59.265369 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" podUID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": read tcp 10.133.0.2:52020->10.133.0.59:15021: read: connection reset by peer" Jul 20 05:30:59.793851 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:59.793812 2576 generic.go:356] "Generic (PLEG): container finished" podID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerID="031415adbcd35b92bffadda498bd134cfe92817805713d1d884e10393377f836" exitCode=0 Jul 20 05:30:59.794028 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:59.793869 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" event={"ID":"b7c270fc-30ef-4091-ad7c-d5d53ab5a832","Type":"ContainerDied","Data":"031415adbcd35b92bffadda498bd134cfe92817805713d1d884e10393377f836"} Jul 20 05:30:59.794308 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:59.794290 2576 scope.go:117] "RemoveContainer" containerID="031415adbcd35b92bffadda498bd134cfe92817805713d1d884e10393377f836" Jul 20 05:30:59.795339 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:59.795311 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:30:59.795395 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:59.795381 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:30:59.795428 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:30:59.795411 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 05:31:00.801801 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:00.801764 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" event={"ID":"b7c270fc-30ef-4091-ad7c-d5d53ab5a832","Type":"ContainerStarted","Data":"5cdac7d90dc9b53729fbd0da95cffd24dd2107dec4500f1c1a45a247e7b7f18c"} Jul 20 05:31:01.395843 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:01.395806 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:31:01.396096 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:01.396072 2576 patch_prober.go:28] interesting pod/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 20 05:31:01.396151 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:01.396125 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" podUID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 20 05:31:02.396150 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:02.396115 2576 patch_prober.go:28] interesting pod/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 20 05:31:02.396549 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:02.396172 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" podUID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 20 05:31:03.396015 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:03.395981 2576 patch_prober.go:28] interesting pod/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 20 05:31:03.396240 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:03.396034 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" podUID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 20 05:31:04.395204 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:04.395171 2576 patch_prober.go:28] interesting pod/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 20 05:31:04.395405 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:04.395230 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" podUID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 20 05:31:05.399303 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:05.399273 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:31:05.399735 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:05.399719 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:31:05.400392 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:05.400373 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:31:19.974988 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:19.974922 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:24.637888 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:24.637841 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:24.783856 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:24.783812 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h"] Jul 20 05:31:24.784145 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:24.784118 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" podUID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerName="istio-proxy" containerID="cri-o://5cdac7d90dc9b53729fbd0da95cffd24dd2107dec4500f1c1a45a247e7b7f18c" gracePeriod=30 Jul 20 05:31:24.806201 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:24.806173 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:25.033612 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:25.033573 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:25.268891 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:25.268861 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:25.367262 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:25.367178 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:25.482993 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:25.482964 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:25.806208 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:25.806173 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:25.905388 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:25.905353 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:29.979331 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:29.978864 2576 generic.go:356] "Generic (PLEG): container finished" podID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerID="5cdac7d90dc9b53729fbd0da95cffd24dd2107dec4500f1c1a45a247e7b7f18c" exitCode=0 Jul 20 05:31:29.979331 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:29.978931 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" event={"ID":"b7c270fc-30ef-4091-ad7c-d5d53ab5a832","Type":"ContainerDied","Data":"5cdac7d90dc9b53729fbd0da95cffd24dd2107dec4500f1c1a45a247e7b7f18c"} Jul 20 05:31:29.979331 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:29.979010 2576 scope.go:117] "RemoveContainer" containerID="031415adbcd35b92bffadda498bd134cfe92817805713d1d884e10393377f836" Jul 20 05:31:30.046907 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.046854 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:31:30.096344 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.096305 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-data\") pod \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " Jul 20 05:31:30.096511 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.096363 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-podinfo\") pod \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " Jul 20 05:31:30.096511 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.096417 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-envoy\") pod \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " Jul 20 05:31:30.096511 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.096486 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-workload-socket\") pod \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " Jul 20 05:31:30.096674 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.096521 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-workload-certs\") pod \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " Jul 20 05:31:30.096674 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.096555 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-credential-socket\") pod \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " Jul 20 05:31:30.096674 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.096635 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-token\") pod \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " Jul 20 05:31:30.096833 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.096691 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istiod-ca-cert\") pod \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " Jul 20 05:31:30.096833 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.096749 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsrww\" (UniqueName: \"kubernetes.io/projected/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-kube-api-access-wsrww\") pod \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\" (UID: \"b7c270fc-30ef-4091-ad7c-d5d53ab5a832\") " Jul 20 05:31:30.096968 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.096835 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b7c270fc-30ef-4091-ad7c-d5d53ab5a832" (UID: "b7c270fc-30ef-4091-ad7c-d5d53ab5a832"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:31:30.097331 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.097188 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-data" (OuterVolumeSpecName: "istio-data") pod "b7c270fc-30ef-4091-ad7c-d5d53ab5a832" (UID: "b7c270fc-30ef-4091-ad7c-d5d53ab5a832"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:31:30.097331 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.097304 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b7c270fc-30ef-4091-ad7c-d5d53ab5a832" (UID: "b7c270fc-30ef-4091-ad7c-d5d53ab5a832"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:31:30.097530 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.097376 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-workload-socket\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:31:30.097530 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.097479 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b7c270fc-30ef-4091-ad7c-d5d53ab5a832" (UID: "b7c270fc-30ef-4091-ad7c-d5d53ab5a832"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:31:30.097645 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.097509 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b7c270fc-30ef-4091-ad7c-d5d53ab5a832" (UID: "b7c270fc-30ef-4091-ad7c-d5d53ab5a832"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:31:30.098969 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.098916 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b7c270fc-30ef-4091-ad7c-d5d53ab5a832" (UID: "b7c270fc-30ef-4091-ad7c-d5d53ab5a832"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 05:31:30.099157 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.099137 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b7c270fc-30ef-4091-ad7c-d5d53ab5a832" (UID: "b7c270fc-30ef-4091-ad7c-d5d53ab5a832"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 05:31:30.099459 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.099430 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-kube-api-access-wsrww" (OuterVolumeSpecName: "kube-api-access-wsrww") pod "b7c270fc-30ef-4091-ad7c-d5d53ab5a832" (UID: "b7c270fc-30ef-4091-ad7c-d5d53ab5a832"). InnerVolumeSpecName "kube-api-access-wsrww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:31:30.099555 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.099470 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7c270fc-30ef-4091-ad7c-d5d53ab5a832" (UID: "b7c270fc-30ef-4091-ad7c-d5d53ab5a832"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:31:30.198215 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.198185 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-token\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:31:30.198215 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.198210 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istiod-ca-cert\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:31:30.198215 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.198220 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsrww\" (UniqueName: \"kubernetes.io/projected/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-kube-api-access-wsrww\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:31:30.198445 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.198229 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-data\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:31:30.198445 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.198238 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-podinfo\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:31:30.198445 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.198246 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-istio-envoy\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:31:30.198445 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.198253 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-workload-certs\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:31:30.198445 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.198261 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7c270fc-30ef-4091-ad7c-d5d53ab5a832-credential-socket\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:31:30.986264 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.986234 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" Jul 20 05:31:30.986264 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.986255 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h" event={"ID":"b7c270fc-30ef-4091-ad7c-d5d53ab5a832","Type":"ContainerDied","Data":"993ae3bee29e010c0c1b4ea7d44781646de7954f28291817d4c81774107951ad"} Jul 20 05:31:30.986752 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:30.986320 2576 scope.go:117] "RemoveContainer" containerID="5cdac7d90dc9b53729fbd0da95cffd24dd2107dec4500f1c1a45a247e7b7f18c" Jul 20 05:31:31.019297 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:31.019263 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h"] Jul 20 05:31:31.026865 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:31.026838 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-50b57cd2-openshift-default-7675b94f99-x5d7h"] Jul 20 05:31:31.177681 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:31.177648 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:31.282143 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:31.282058 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:31.659999 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:31.655916 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" path="/var/lib/kubelet/pods/b7c270fc-30ef-4091-ad7c-d5d53ab5a832/volumes" Jul 20 05:31:31.697172 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:31.697141 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:31.799989 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:31.799916 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:31.897950 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:31.897898 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:31:56.126252 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:56.126216 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z"] Jul 20 05:31:56.126680 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:56.126666 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerName="istio-proxy" Jul 20 05:31:56.126723 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:56.126690 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerName="istio-proxy" Jul 20 05:31:56.126774 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:56.126765 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerName="istio-proxy" Jul 20 05:31:56.129825 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:56.129807 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z" Jul 20 05:31:56.132502 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:56.132481 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:31:56.137812 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:56.137790 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z"] Jul 20 05:31:56.235172 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:56.235137 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6373feb4-1d1d-406b-a68f-8c0a41acc50b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z\" (UID: \"6373feb4-1d1d-406b-a68f-8c0a41acc50b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z" Jul 20 05:31:56.336574 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:56.336545 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6373feb4-1d1d-406b-a68f-8c0a41acc50b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z\" (UID: \"6373feb4-1d1d-406b-a68f-8c0a41acc50b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z" Jul 20 05:31:56.337292 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:56.337272 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6373feb4-1d1d-406b-a68f-8c0a41acc50b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z\" (UID: \"6373feb4-1d1d-406b-a68f-8c0a41acc50b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z" Jul 20 05:31:56.444041 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:56.443949 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z" Jul 20 05:31:56.786564 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:56.786535 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z"] Jul 20 05:31:56.787604 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:31:56.787565 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6373feb4_1d1d_406b_a68f_8c0a41acc50b.slice/crio-edbcd2aabf90ea783752fcf4599e04303670caa3a09ec5b42980b150e980b570 WatchSource:0}: Error finding container edbcd2aabf90ea783752fcf4599e04303670caa3a09ec5b42980b150e980b570: Status 404 returned error can't find the container with id edbcd2aabf90ea783752fcf4599e04303670caa3a09ec5b42980b150e980b570 Jul 20 05:31:57.140150 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:57.140113 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z" event={"ID":"6373feb4-1d1d-406b-a68f-8c0a41acc50b","Type":"ContainerStarted","Data":"f9a37bcc94c8e0a6f1ef24c331ab12a62671ac7e493c3d8491ae6866f40a48bb"} Jul 20 05:31:57.140533 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:57.140170 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z" event={"ID":"6373feb4-1d1d-406b-a68f-8c0a41acc50b","Type":"ContainerStarted","Data":"edbcd2aabf90ea783752fcf4599e04303670caa3a09ec5b42980b150e980b570"} Jul 20 05:31:57.157050 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:57.157003 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z" podStartSLOduration=1.156989812 podStartE2EDuration="1.156989812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:31:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:31:57.154143783 +0000 UTC m=+1960.200670691" watchObservedRunningTime="2026-07-20 05:31:57.156989812 +0000 UTC m=+1960.203516742" Jul 20 05:31:58.150316 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:58.150281 2576 generic.go:356] "Generic (PLEG): container finished" podID="6373feb4-1d1d-406b-a68f-8c0a41acc50b" containerID="f9a37bcc94c8e0a6f1ef24c331ab12a62671ac7e493c3d8491ae6866f40a48bb" exitCode=0 Jul 20 05:31:58.150316 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:58.150317 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z" event={"ID":"6373feb4-1d1d-406b-a68f-8c0a41acc50b","Type":"ContainerDied","Data":"f9a37bcc94c8e0a6f1ef24c331ab12a62671ac7e493c3d8491ae6866f40a48bb"} Jul 20 05:31:59.288373 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:59.288348 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z" Jul 20 05:31:59.364332 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:59.364293 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6373feb4-1d1d-406b-a68f-8c0a41acc50b-maas-api-service-ca\") pod \"6373feb4-1d1d-406b-a68f-8c0a41acc50b\" (UID: \"6373feb4-1d1d-406b-a68f-8c0a41acc50b\") " Jul 20 05:31:59.364611 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:59.364589 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6373feb4-1d1d-406b-a68f-8c0a41acc50b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6373feb4-1d1d-406b-a68f-8c0a41acc50b" (UID: "6373feb4-1d1d-406b-a68f-8c0a41acc50b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:31:59.465327 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:31:59.465255 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6373feb4-1d1d-406b-a68f-8c0a41acc50b-maas-api-service-ca\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:32:00.163292 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:00.163264 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z" Jul 20 05:32:00.163451 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:00.163266 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z" event={"ID":"6373feb4-1d1d-406b-a68f-8c0a41acc50b","Type":"ContainerDied","Data":"edbcd2aabf90ea783752fcf4599e04303670caa3a09ec5b42980b150e980b570"} Jul 20 05:32:00.163451 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:00.163399 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="edbcd2aabf90ea783752fcf4599e04303670caa3a09ec5b42980b150e980b570" Jul 20 05:32:06.538199 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:06.538139 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:11.202629 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:11.202589 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z"] Jul 20 05:32:11.205387 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:11.205347 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:11.206189 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:11.206169 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-be1ac6ef-n7k8z"] Jul 20 05:32:11.350438 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:11.350390 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:11.573999 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:11.573942 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:11.653945 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:11.653913 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6373feb4-1d1d-406b-a68f-8c0a41acc50b" path="/var/lib/kubelet/pods/6373feb4-1d1d-406b-a68f-8c0a41acc50b/volumes" Jul 20 05:32:11.812463 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:11.812429 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:12.233853 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:12.233818 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:12.934929 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:12.934898 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:14.269185 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:14.269149 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:14.350849 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:14.350782 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:14.424905 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:14.424848 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:14.887024 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:14.886969 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:14.988412 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:14.988383 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:15.088946 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:15.088918 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:39.135899 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:39.135868 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:39.237630 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:39.237576 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:40.074758 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.074723 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq"] Jul 20 05:32:40.075270 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.075250 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6373feb4-1d1d-406b-a68f-8c0a41acc50b" containerName="revoke-keys" Jul 20 05:32:40.075376 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.075272 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6373feb4-1d1d-406b-a68f-8c0a41acc50b" containerName="revoke-keys" Jul 20 05:32:40.075376 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.075288 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerName="istio-proxy" Jul 20 05:32:40.075376 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.075295 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerName="istio-proxy" Jul 20 05:32:40.075527 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.075395 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6373feb4-1d1d-406b-a68f-8c0a41acc50b" containerName="revoke-keys" Jul 20 05:32:40.075527 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.075415 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7c270fc-30ef-4091-ad7c-d5d53ab5a832" containerName="istio-proxy" Jul 20 05:32:40.078806 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.078786 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq" Jul 20 05:32:40.083459 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.083433 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:32:40.084689 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.084668 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq"] Jul 20 05:32:40.137322 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.137290 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e76c54b-32eb-4c8b-813c-242586cb2dd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq\" (UID: \"3e76c54b-32eb-4c8b-813c-242586cb2dd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq" Jul 20 05:32:40.238631 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.238599 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e76c54b-32eb-4c8b-813c-242586cb2dd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq\" (UID: \"3e76c54b-32eb-4c8b-813c-242586cb2dd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq" Jul 20 05:32:40.239241 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.239221 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e76c54b-32eb-4c8b-813c-242586cb2dd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq\" (UID: \"3e76c54b-32eb-4c8b-813c-242586cb2dd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq" Jul 20 05:32:40.391210 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.391134 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq" Jul 20 05:32:40.526435 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.526411 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq"] Jul 20 05:32:40.529584 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:32:40.529548 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e76c54b_32eb_4c8b_813c_242586cb2dd7.slice/crio-50ebe4a63293d77acce45c938ed293fb14e5827789407174a4e5bad1790ea9cd WatchSource:0}: Error finding container 50ebe4a63293d77acce45c938ed293fb14e5827789407174a4e5bad1790ea9cd: Status 404 returned error can't find the container with id 50ebe4a63293d77acce45c938ed293fb14e5827789407174a4e5bad1790ea9cd Jul 20 05:32:40.718175 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.718071 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:40.819092 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:40.819059 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:41.404997 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:41.404948 2576 generic.go:356] "Generic (PLEG): container finished" podID="3e76c54b-32eb-4c8b-813c-242586cb2dd7" containerID="5741efacddc3a16f95b00bacb4b5a08275fe7afb160903d85a01e8ffa089aef8" exitCode=0 Jul 20 05:32:41.405429 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:41.405048 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq" event={"ID":"3e76c54b-32eb-4c8b-813c-242586cb2dd7","Type":"ContainerDied","Data":"5741efacddc3a16f95b00bacb4b5a08275fe7afb160903d85a01e8ffa089aef8"} Jul 20 05:32:41.405429 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:41.405098 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq" event={"ID":"3e76c54b-32eb-4c8b-813c-242586cb2dd7","Type":"ContainerStarted","Data":"50ebe4a63293d77acce45c938ed293fb14e5827789407174a4e5bad1790ea9cd"} Jul 20 05:32:42.549746 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:42.549721 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq" Jul 20 05:32:42.661134 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:42.661106 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e76c54b-32eb-4c8b-813c-242586cb2dd7-maas-api-service-ca\") pod \"3e76c54b-32eb-4c8b-813c-242586cb2dd7\" (UID: \"3e76c54b-32eb-4c8b-813c-242586cb2dd7\") " Jul 20 05:32:42.661457 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:42.661413 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e76c54b-32eb-4c8b-813c-242586cb2dd7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3e76c54b-32eb-4c8b-813c-242586cb2dd7" (UID: "3e76c54b-32eb-4c8b-813c-242586cb2dd7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:32:42.762750 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:42.762711 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e76c54b-32eb-4c8b-813c-242586cb2dd7-maas-api-service-ca\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:32:43.418515 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:43.418486 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq" Jul 20 05:32:43.418686 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:43.418485 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq" event={"ID":"3e76c54b-32eb-4c8b-813c-242586cb2dd7","Type":"ContainerDied","Data":"50ebe4a63293d77acce45c938ed293fb14e5827789407174a4e5bad1790ea9cd"} Jul 20 05:32:43.418686 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:43.418617 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50ebe4a63293d77acce45c938ed293fb14e5827789407174a4e5bad1790ea9cd" Jul 20 05:32:50.547504 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:50.547468 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:55.144755 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:55.144722 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:55.147292 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:55.147263 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq"] Jul 20 05:32:55.149877 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:55.149850 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-defc77b8-9p2nq"] Jul 20 05:32:55.653898 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:55.653866 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e76c54b-32eb-4c8b-813c-242586cb2dd7" path="/var/lib/kubelet/pods/3e76c54b-32eb-4c8b-813c-242586cb2dd7/volumes" Jul 20 05:32:56.354763 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:56.354735 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:56.593038 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:56.593002 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:56.818217 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:56.818184 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:57.134013 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:57.133915 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:57.231098 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:57.231069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:59.510177 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:59.510142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:59.613234 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:59.613193 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:59.683916 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:59.683878 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:32:59.784478 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:32:59.784400 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:00.162746 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:00.162667 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:00.263610 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:00.263581 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:24.397517 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:24.397469 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x"] Jul 20 05:33:24.398051 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:24.397931 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e76c54b-32eb-4c8b-813c-242586cb2dd7" containerName="revoke-keys" Jul 20 05:33:24.398051 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:24.397946 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e76c54b-32eb-4c8b-813c-242586cb2dd7" containerName="revoke-keys" Jul 20 05:33:24.398129 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:24.398089 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e76c54b-32eb-4c8b-813c-242586cb2dd7" containerName="revoke-keys" Jul 20 05:33:24.401282 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:24.401263 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x" Jul 20 05:33:24.403673 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:24.403654 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:33:24.410628 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:24.409374 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x"] Jul 20 05:33:24.453635 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:24.453602 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x\" (UID: \"3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x" Jul 20 05:33:24.554974 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:24.554933 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x\" (UID: \"3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x" Jul 20 05:33:24.555566 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:24.555547 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x\" (UID: \"3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x" Jul 20 05:33:24.714793 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:24.714706 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x" Jul 20 05:33:24.862141 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:24.862109 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x"] Jul 20 05:33:24.864433 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:33:24.864398 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b8d5bcd_2be3_491f_b7ca_cf0dcb580d5c.slice/crio-596a8631f1a9065fc9c93573c29631ccea528e8549778e4c823fdd1379f05fb4 WatchSource:0}: Error finding container 596a8631f1a9065fc9c93573c29631ccea528e8549778e4c823fdd1379f05fb4: Status 404 returned error can't find the container with id 596a8631f1a9065fc9c93573c29631ccea528e8549778e4c823fdd1379f05fb4 Jul 20 05:33:25.663406 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:25.663369 2576 generic.go:356] "Generic (PLEG): container finished" podID="3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c" containerID="0fc42371eda9c1198f3d19aa47865ba7341cff3a731e29b0272abfffda9c59e2" exitCode=0 Jul 20 05:33:25.663820 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:25.663461 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x" event={"ID":"3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c","Type":"ContainerDied","Data":"0fc42371eda9c1198f3d19aa47865ba7341cff3a731e29b0272abfffda9c59e2"} Jul 20 05:33:25.663820 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:25.663525 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x" event={"ID":"3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c","Type":"ContainerStarted","Data":"596a8631f1a9065fc9c93573c29631ccea528e8549778e4c823fdd1379f05fb4"} Jul 20 05:33:26.808869 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:26.808846 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x" Jul 20 05:33:26.877738 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:26.877704 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c-maas-api-service-ca\") pod \"3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c\" (UID: \"3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c\") " Jul 20 05:33:26.878091 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:26.878068 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c" (UID: "3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:33:26.979425 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:26.979350 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c-maas-api-service-ca\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:33:27.677299 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:27.677267 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x" event={"ID":"3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c","Type":"ContainerDied","Data":"596a8631f1a9065fc9c93573c29631ccea528e8549778e4c823fdd1379f05fb4"} Jul 20 05:33:27.677299 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:27.677294 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x" Jul 20 05:33:27.677464 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:27.677311 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="596a8631f1a9065fc9c93573c29631ccea528e8549778e4c823fdd1379f05fb4" Jul 20 05:33:34.836544 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:34.836510 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:39.464246 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:39.464162 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x"] Jul 20 05:33:39.465645 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:39.465618 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:39.466752 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:39.466733 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-3a653f83-zkz6x"] Jul 20 05:33:39.611727 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:39.611699 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:39.654033 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:39.653994 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c" path="/var/lib/kubelet/pods/3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c/volumes" Jul 20 05:33:39.848932 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:39.848898 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:40.306777 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:40.306744 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:40.411243 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:40.411211 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:40.506830 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:40.506786 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:41.970583 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:41.970552 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:47.623391 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:47.623359 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:47.685417 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:47.685383 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:47.760377 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:47.760344 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:47.857614 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:47.857579 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:47.975168 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:47.975087 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:33:48.357521 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:33:48.357488 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:14.351596 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:14.351559 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:15.009931 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:15.009895 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:15.064850 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:15.064807 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:17.890293 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:17.890248 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:34:17.891812 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:17.891788 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:34:17.894639 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:17.894621 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:34:17.896209 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:17.896191 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:34:20.613134 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:20.613096 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:20.720487 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:20.720457 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:20.813386 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:20.813358 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:36.674048 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:36.674012 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:42.338003 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:42.337944 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:42.468929 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:42.468892 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:42.568004 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:42.567944 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:53.075422 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:53.075387 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:53.923187 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:53.923152 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:55.567648 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:55.567616 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:59.802974 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:59.802922 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:34:59.906346 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:34:59.906315 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:00.004825 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:00.004779 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:00.102414 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:00.102333 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:00.896440 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:00.896404 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:00.992133 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:00.992096 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:01.092211 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:01.092179 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:02.756223 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:02.756195 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:08.791108 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:08.791076 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:08.895607 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:08.895577 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:09.516837 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:09.516797 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:15.735505 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:15.735477 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:15.837905 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:15.837870 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:15.971352 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:15.971318 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:16.304561 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:16.304526 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:16.627809 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:16.627720 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:19.721278 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:19.721244 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:19.810766 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:19.810734 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:19.890931 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:19.890876 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:20.007372 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:20.007339 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:20.601925 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:20.601893 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:20.698390 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:20.698357 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:24.255199 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:24.255165 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:24.943805 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:24.943770 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:25.044931 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:25.044894 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:25.402527 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:25.402494 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:31.048355 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.048315 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:31.148535 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.148495 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:31.442060 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.441966 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss"] Jul 20 05:35:31.442386 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.442372 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c" containerName="revoke-keys" Jul 20 05:35:31.442386 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.442386 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c" containerName="revoke-keys" Jul 20 05:35:31.442519 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.442484 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b8d5bcd-2be3-491f-b7ca-cf0dcb580d5c" containerName="revoke-keys" Jul 20 05:35:31.445703 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.445681 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss" Jul 20 05:35:31.448281 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.448257 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:35:31.454437 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.453898 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss"] Jul 20 05:35:31.467192 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.467171 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27b4da34-0370-4cea-8645-fe7840726b15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d01a7226-879ss\" (UID: \"27b4da34-0370-4cea-8645-fe7840726b15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss" Jul 20 05:35:31.568780 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.568744 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27b4da34-0370-4cea-8645-fe7840726b15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d01a7226-879ss\" (UID: \"27b4da34-0370-4cea-8645-fe7840726b15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss" Jul 20 05:35:31.569355 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.569331 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27b4da34-0370-4cea-8645-fe7840726b15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d01a7226-879ss\" (UID: \"27b4da34-0370-4cea-8645-fe7840726b15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss" Jul 20 05:35:31.757546 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.757515 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss" Jul 20 05:35:31.898162 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:31.898135 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss"] Jul 20 05:35:31.899297 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:35:31.899267 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27b4da34_0370_4cea_8645_fe7840726b15.slice/crio-aba38217e4ec6df39bde84bca044732ed13acec8b70d02240c3635ff6d793ec2 WatchSource:0}: Error finding container aba38217e4ec6df39bde84bca044732ed13acec8b70d02240c3635ff6d793ec2: Status 404 returned error can't find the container with id aba38217e4ec6df39bde84bca044732ed13acec8b70d02240c3635ff6d793ec2 Jul 20 05:35:32.413849 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:32.413750 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss" event={"ID":"27b4da34-0370-4cea-8645-fe7840726b15","Type":"ContainerStarted","Data":"2501e0e92d9aba85d8860420728bcee9f83419a813ece3a2adc4273eb1b2926f"} Jul 20 05:35:32.413849 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:32.413807 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss" event={"ID":"27b4da34-0370-4cea-8645-fe7840726b15","Type":"ContainerStarted","Data":"aba38217e4ec6df39bde84bca044732ed13acec8b70d02240c3635ff6d793ec2"} Jul 20 05:35:32.429820 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:32.429766 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss" podStartSLOduration=1.429751805 podStartE2EDuration="1.429751805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:35:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:35:32.426605234 +0000 UTC m=+2175.473132140" watchObservedRunningTime="2026-07-20 05:35:32.429751805 +0000 UTC m=+2175.476278711" Jul 20 05:35:33.420437 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:33.420402 2576 generic.go:356] "Generic (PLEG): container finished" podID="27b4da34-0370-4cea-8645-fe7840726b15" containerID="2501e0e92d9aba85d8860420728bcee9f83419a813ece3a2adc4273eb1b2926f" exitCode=0 Jul 20 05:35:33.420813 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:33.420495 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss" event={"ID":"27b4da34-0370-4cea-8645-fe7840726b15","Type":"ContainerDied","Data":"2501e0e92d9aba85d8860420728bcee9f83419a813ece3a2adc4273eb1b2926f"} Jul 20 05:35:34.570474 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:34.570448 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss" Jul 20 05:35:34.697895 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:34.697816 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27b4da34-0370-4cea-8645-fe7840726b15-maas-api-service-ca\") pod \"27b4da34-0370-4cea-8645-fe7840726b15\" (UID: \"27b4da34-0370-4cea-8645-fe7840726b15\") " Jul 20 05:35:34.698176 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:34.698156 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27b4da34-0370-4cea-8645-fe7840726b15-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "27b4da34-0370-4cea-8645-fe7840726b15" (UID: "27b4da34-0370-4cea-8645-fe7840726b15"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:35:34.799368 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:34.799333 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27b4da34-0370-4cea-8645-fe7840726b15-maas-api-service-ca\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:35:35.433430 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:35.433399 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss" Jul 20 05:35:35.433595 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:35.433434 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss" event={"ID":"27b4da34-0370-4cea-8645-fe7840726b15","Type":"ContainerDied","Data":"aba38217e4ec6df39bde84bca044732ed13acec8b70d02240c3635ff6d793ec2"} Jul 20 05:35:35.433595 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:35.433513 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aba38217e4ec6df39bde84bca044732ed13acec8b70d02240c3635ff6d793ec2" Jul 20 05:35:41.837528 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:41.837496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:46.501710 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:46.501671 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:46.508973 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:46.508918 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss"] Jul 20 05:35:46.511422 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:46.511402 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d01a7226-879ss"] Jul 20 05:35:47.653939 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:47.653895 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27b4da34-0370-4cea-8645-fe7840726b15" path="/var/lib/kubelet/pods/27b4da34-0370-4cea-8645-fe7840726b15/volumes" Jul 20 05:35:47.767271 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:47.767238 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:52.551543 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:52.551516 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:52.616655 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:52.616623 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:52.820790 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:52.820707 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:53.065976 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:53.065909 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:53.188760 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:53.188682 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:53.288054 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:53.288019 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:53.410550 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:53.410515 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:59.138584 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:59.138547 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:59.191136 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:59.191108 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:59.268413 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:59.268280 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:59.361271 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:59.361222 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:35:59.487324 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:35:59.487293 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:05.183268 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:05.183234 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:05.283740 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:05.283708 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:12.148932 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:12.148896 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:12.470180 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:12.470091 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:12.547657 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:12.547626 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:15.022449 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:15.022417 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:15.113471 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:15.113433 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:15.178775 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:15.178744 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:15.540343 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:15.540306 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:15.680620 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:15.680589 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:15.781196 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:15.781163 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:19.580827 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:19.580785 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:25.639033 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:25.638992 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:42.832935 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:42.832836 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:36:48.008354 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.008317 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b"] Jul 20 05:36:48.008735 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.008722 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27b4da34-0370-4cea-8645-fe7840726b15" containerName="revoke-keys" Jul 20 05:36:48.008784 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.008736 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="27b4da34-0370-4cea-8645-fe7840726b15" containerName="revoke-keys" Jul 20 05:36:48.008837 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.008820 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="27b4da34-0370-4cea-8645-fe7840726b15" containerName="revoke-keys" Jul 20 05:36:48.012256 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.012232 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b" Jul 20 05:36:48.015256 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.015080 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:36:48.018853 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.018827 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b"] Jul 20 05:36:48.191697 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.191660 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c35fd850-c789-40f4-9f1c-15c7971c659a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b\" (UID: \"c35fd850-c789-40f4-9f1c-15c7971c659a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b" Jul 20 05:36:48.292375 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.292284 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c35fd850-c789-40f4-9f1c-15c7971c659a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b\" (UID: \"c35fd850-c789-40f4-9f1c-15c7971c659a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b" Jul 20 05:36:48.293038 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.293006 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c35fd850-c789-40f4-9f1c-15c7971c659a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b\" (UID: \"c35fd850-c789-40f4-9f1c-15c7971c659a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b" Jul 20 05:36:48.325421 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.325394 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b" Jul 20 05:36:48.464852 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.464809 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b"] Jul 20 05:36:48.468635 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:36:48.468600 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc35fd850_c789_40f4_9f1c_15c7971c659a.slice/crio-35a93f7e06faf66e98ed5bb71b4d0da324aa58a5bc04b301ac2aa124cff21f5e WatchSource:0}: Error finding container 35a93f7e06faf66e98ed5bb71b4d0da324aa58a5bc04b301ac2aa124cff21f5e: Status 404 returned error can't find the container with id 35a93f7e06faf66e98ed5bb71b4d0da324aa58a5bc04b301ac2aa124cff21f5e Jul 20 05:36:48.862267 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.862226 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b" event={"ID":"c35fd850-c789-40f4-9f1c-15c7971c659a","Type":"ContainerStarted","Data":"130af06b015c0a714752589b374ac33109243e970579c026948bf5a46f1dd28e"} Jul 20 05:36:48.862426 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.862289 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b" event={"ID":"c35fd850-c789-40f4-9f1c-15c7971c659a","Type":"ContainerStarted","Data":"35a93f7e06faf66e98ed5bb71b4d0da324aa58a5bc04b301ac2aa124cff21f5e"} Jul 20 05:36:48.878579 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:48.878529 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b" podStartSLOduration=1.878515169 podStartE2EDuration="1.878515169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:36:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:36:48.87605829 +0000 UTC m=+2251.922585212" watchObservedRunningTime="2026-07-20 05:36:48.878515169 +0000 UTC m=+2251.925042074" Jul 20 05:36:49.869263 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:49.869232 2576 generic.go:356] "Generic (PLEG): container finished" podID="c35fd850-c789-40f4-9f1c-15c7971c659a" containerID="130af06b015c0a714752589b374ac33109243e970579c026948bf5a46f1dd28e" exitCode=0 Jul 20 05:36:49.869644 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:49.869287 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b" event={"ID":"c35fd850-c789-40f4-9f1c-15c7971c659a","Type":"ContainerDied","Data":"130af06b015c0a714752589b374ac33109243e970579c026948bf5a46f1dd28e"} Jul 20 05:36:51.011145 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:51.011124 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b" Jul 20 05:36:51.011544 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:51.011523 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c35fd850-c789-40f4-9f1c-15c7971c659a-maas-api-service-ca\") pod \"c35fd850-c789-40f4-9f1c-15c7971c659a\" (UID: \"c35fd850-c789-40f4-9f1c-15c7971c659a\") " Jul 20 05:36:51.011898 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:51.011877 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c35fd850-c789-40f4-9f1c-15c7971c659a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c35fd850-c789-40f4-9f1c-15c7971c659a" (UID: "c35fd850-c789-40f4-9f1c-15c7971c659a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:36:51.011967 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:51.011942 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c35fd850-c789-40f4-9f1c-15c7971c659a-maas-api-service-ca\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:36:51.882345 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:51.882320 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b" Jul 20 05:36:51.882503 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:51.882315 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b" event={"ID":"c35fd850-c789-40f4-9f1c-15c7971c659a","Type":"ContainerDied","Data":"35a93f7e06faf66e98ed5bb71b4d0da324aa58a5bc04b301ac2aa124cff21f5e"} Jul 20 05:36:51.882503 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:36:51.882444 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35a93f7e06faf66e98ed5bb71b4d0da324aa58a5bc04b301ac2aa124cff21f5e" Jul 20 05:37:03.533680 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:03.533650 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:08.089096 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:08.089059 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b"] Jul 20 05:37:08.091572 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:08.091550 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-77b596d2-s2m5b"] Jul 20 05:37:08.106397 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:08.106345 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:09.653838 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:09.653803 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c35fd850-c789-40f4-9f1c-15c7971c659a" path="/var/lib/kubelet/pods/c35fd850-c789-40f4-9f1c-15c7971c659a/volumes" Jul 20 05:37:10.421103 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:10.421068 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:15.455412 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:15.455377 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:39.971767 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:39.971729 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:40.070531 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:40.070495 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:44.376521 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:44.376488 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:44.474141 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:44.474106 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:51.869619 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:51.869559 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:52.428319 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:52.428289 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:58.147657 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:58.147615 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:58.228583 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:58.228552 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:58.435719 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:58.435633 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:58.695008 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:58.694903 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:59.714774 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:59.714740 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:59.818528 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:59.818496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:37:59.928558 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:37:59.928529 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:00.692808 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:00.692767 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:00.780866 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:00.780811 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:00.880321 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:00.880288 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:01.072098 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:01.072065 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:02.311625 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:02.311592 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:08.205432 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:08.205387 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:08.261996 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:08.261967 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:08.895037 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:08.895005 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:15.023630 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:15.023598 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:15.129387 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:15.129333 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:15.221782 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:15.221747 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:18.997322 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:18.997291 2576 scope.go:117] "RemoveContainer" containerID="f9a37bcc94c8e0a6f1ef24c331ab12a62671ac7e493c3d8491ae6866f40a48bb" Jul 20 05:38:25.734380 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:25.734346 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:26.079259 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:26.079173 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:47.032556 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:47.032518 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:53.133655 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:53.133622 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:53.234913 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:53.234879 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:53.520028 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:53.519993 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:53.844322 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:53.844249 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:54.202565 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:54.202491 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:56.828189 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:56.828151 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:56.898294 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:56.898259 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:56.997568 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:56.997532 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:57.095062 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:57.094974 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:57.572027 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:57.571993 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:38:57.672669 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:38:57.672641 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:01.349708 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:01.349670 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:01.700770 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:01.700698 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:02.016549 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:02.016517 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:02.273812 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:02.273717 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:04.841763 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:04.841722 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:04.941466 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:04.941429 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:05.212173 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:05.212101 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:05.310067 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:05.310020 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:05.629541 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:05.629504 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:05.692156 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:05.692117 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:09.345952 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:09.345918 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:17.933773 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:17.933746 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:39:17.937762 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:17.937741 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:39:17.937900 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:17.937880 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:39:17.942076 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:17.942058 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:39:18.293351 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:18.293315 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:18.394365 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:18.394332 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:19.019566 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:19.019539 2576 scope.go:117] "RemoveContainer" containerID="5741efacddc3a16f95b00bacb4b5a08275fe7afb160903d85a01e8ffa089aef8" Jul 20 05:39:19.554687 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:19.554655 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:26.384460 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:26.384422 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:26.489431 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:26.489399 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:30.498687 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:30.498653 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:39:40.751035 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:39:40.750939 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:19.041878 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:19.041843 2576 scope.go:117] "RemoveContainer" containerID="0fc42371eda9c1198f3d19aa47865ba7341cff3a731e29b0272abfffda9c59e2" Jul 20 05:40:23.832670 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:23.832632 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:24.758819 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:24.758782 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:24.860502 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:24.860470 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd"] Jul 20 05:40:24.860997 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:24.860916 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c35fd850-c789-40f4-9f1c-15c7971c659a" containerName="revoke-keys" Jul 20 05:40:24.860997 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:24.860928 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c35fd850-c789-40f4-9f1c-15c7971c659a" containerName="revoke-keys" Jul 20 05:40:24.861127 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:24.861021 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c35fd850-c789-40f4-9f1c-15c7971c659a" containerName="revoke-keys" Jul 20 05:40:24.863720 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:24.863699 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd" Jul 20 05:40:24.866185 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:24.866163 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:40:24.872002 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:24.871384 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd"] Jul 20 05:40:24.990716 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:24.990688 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e59c646-4b35-4474-8a0d-4dfa6970be98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd\" (UID: \"3e59c646-4b35-4474-8a0d-4dfa6970be98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd" Jul 20 05:40:25.091532 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:25.091433 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e59c646-4b35-4474-8a0d-4dfa6970be98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd\" (UID: \"3e59c646-4b35-4474-8a0d-4dfa6970be98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd" Jul 20 05:40:25.092175 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:25.092154 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e59c646-4b35-4474-8a0d-4dfa6970be98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd\" (UID: \"3e59c646-4b35-4474-8a0d-4dfa6970be98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd" Jul 20 05:40:25.175516 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:25.175487 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd" Jul 20 05:40:25.337856 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:25.337829 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd"] Jul 20 05:40:25.339371 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:40:25.339340 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e59c646_4b35_4474_8a0d_4dfa6970be98.slice/crio-691c509c0372eb835035685fd94e8e4643a729e61c69772e0691ac265ea61223 WatchSource:0}: Error finding container 691c509c0372eb835035685fd94e8e4643a729e61c69772e0691ac265ea61223: Status 404 returned error can't find the container with id 691c509c0372eb835035685fd94e8e4643a729e61c69772e0691ac265ea61223 Jul 20 05:40:25.439866 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:25.439834 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:25.550508 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:25.550474 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:26.169942 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:26.169910 2576 generic.go:356] "Generic (PLEG): container finished" podID="3e59c646-4b35-4474-8a0d-4dfa6970be98" containerID="26e57db18a0dd1abb9b825a28b647d195192d8bb55962b0d64d988734dcf032f" exitCode=0 Jul 20 05:40:26.170295 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:26.169949 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd" event={"ID":"3e59c646-4b35-4474-8a0d-4dfa6970be98","Type":"ContainerDied","Data":"26e57db18a0dd1abb9b825a28b647d195192d8bb55962b0d64d988734dcf032f"} Jul 20 05:40:26.170295 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:26.170050 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd" event={"ID":"3e59c646-4b35-4474-8a0d-4dfa6970be98","Type":"ContainerStarted","Data":"691c509c0372eb835035685fd94e8e4643a729e61c69772e0691ac265ea61223"} Jul 20 05:40:27.314537 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:27.314507 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd" Jul 20 05:40:27.414366 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:27.414336 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e59c646-4b35-4474-8a0d-4dfa6970be98-maas-api-service-ca\") pod \"3e59c646-4b35-4474-8a0d-4dfa6970be98\" (UID: \"3e59c646-4b35-4474-8a0d-4dfa6970be98\") " Jul 20 05:40:27.414678 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:27.414656 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e59c646-4b35-4474-8a0d-4dfa6970be98-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3e59c646-4b35-4474-8a0d-4dfa6970be98" (UID: "3e59c646-4b35-4474-8a0d-4dfa6970be98"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:40:27.515904 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:27.515858 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e59c646-4b35-4474-8a0d-4dfa6970be98-maas-api-service-ca\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:40:28.184632 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:28.184598 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd" Jul 20 05:40:28.184632 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:28.184630 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd" event={"ID":"3e59c646-4b35-4474-8a0d-4dfa6970be98","Type":"ContainerDied","Data":"691c509c0372eb835035685fd94e8e4643a729e61c69772e0691ac265ea61223"} Jul 20 05:40:28.184861 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:28.184675 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="691c509c0372eb835035685fd94e8e4643a729e61c69772e0691ac265ea61223" Jul 20 05:40:35.289591 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:35.289556 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:39.925572 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:39.925529 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:39.942447 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:39.942393 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd"] Jul 20 05:40:39.944988 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:39.944967 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-192e48-ftwsd"] Jul 20 05:40:40.310431 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:40.310396 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:41.653532 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:41.653498 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e59c646-4b35-4474-8a0d-4dfa6970be98" path="/var/lib/kubelet/pods/3e59c646-4b35-4474-8a0d-4dfa6970be98/volumes" Jul 20 05:40:46.353241 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:46.353211 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:46.450898 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:46.450865 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:46.639465 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:46.639386 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:46.863434 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:46.863399 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:46.975130 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:46.975034 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87"] Jul 20 05:40:46.975471 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:46.975457 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e59c646-4b35-4474-8a0d-4dfa6970be98" containerName="revoke-keys" Jul 20 05:40:46.975471 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:46.975471 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e59c646-4b35-4474-8a0d-4dfa6970be98" containerName="revoke-keys" Jul 20 05:40:46.975599 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:46.975541 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e59c646-4b35-4474-8a0d-4dfa6970be98" containerName="revoke-keys" Jul 20 05:40:46.979676 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:46.979652 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87" Jul 20 05:40:46.982491 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:46.982467 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:40:46.986181 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:46.986156 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87"] Jul 20 05:40:47.106941 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:47.106901 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6046668e-5182-408e-82e8-6096b31c76ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87\" (UID: \"6046668e-5182-408e-82e8-6096b31c76ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87" Jul 20 05:40:47.207525 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:47.207484 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6046668e-5182-408e-82e8-6096b31c76ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87\" (UID: \"6046668e-5182-408e-82e8-6096b31c76ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87" Jul 20 05:40:47.208222 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:47.208196 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6046668e-5182-408e-82e8-6096b31c76ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87\" (UID: \"6046668e-5182-408e-82e8-6096b31c76ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87" Jul 20 05:40:47.293299 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:47.293267 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87" Jul 20 05:40:47.430880 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:47.430843 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87"] Jul 20 05:40:47.431568 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:40:47.431538 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6046668e_5182_408e_82e8_6096b31c76ea.slice/crio-ffe2481232285003bb76635685359972ea21983224e75876bfa72d1afc675daf WatchSource:0}: Error finding container ffe2481232285003bb76635685359972ea21983224e75876bfa72d1afc675daf: Status 404 returned error can't find the container with id ffe2481232285003bb76635685359972ea21983224e75876bfa72d1afc675daf Jul 20 05:40:47.681337 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:47.681300 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:47.783435 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:47.783378 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:40:48.302473 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:48.302436 2576 generic.go:356] "Generic (PLEG): container finished" podID="6046668e-5182-408e-82e8-6096b31c76ea" containerID="63dd08cdaf32ec8f0aa9c6e8cffe39aa53b4d53dd95ce38f43859a5f1bdef057" exitCode=0 Jul 20 05:40:48.302641 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:48.302513 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87" event={"ID":"6046668e-5182-408e-82e8-6096b31c76ea","Type":"ContainerDied","Data":"63dd08cdaf32ec8f0aa9c6e8cffe39aa53b4d53dd95ce38f43859a5f1bdef057"} Jul 20 05:40:48.302641 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:48.302556 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87" event={"ID":"6046668e-5182-408e-82e8-6096b31c76ea","Type":"ContainerStarted","Data":"ffe2481232285003bb76635685359972ea21983224e75876bfa72d1afc675daf"} Jul 20 05:40:49.442492 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:49.442468 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87" Jul 20 05:40:49.529559 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:49.529525 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6046668e-5182-408e-82e8-6096b31c76ea-maas-api-service-ca\") pod \"6046668e-5182-408e-82e8-6096b31c76ea\" (UID: \"6046668e-5182-408e-82e8-6096b31c76ea\") " Jul 20 05:40:49.529948 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:49.529922 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6046668e-5182-408e-82e8-6096b31c76ea-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6046668e-5182-408e-82e8-6096b31c76ea" (UID: "6046668e-5182-408e-82e8-6096b31c76ea"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:40:49.630753 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:49.630659 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6046668e-5182-408e-82e8-6096b31c76ea-maas-api-service-ca\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:40:50.315812 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:50.315780 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87" Jul 20 05:40:50.316099 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:50.315817 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87" event={"ID":"6046668e-5182-408e-82e8-6096b31c76ea","Type":"ContainerDied","Data":"ffe2481232285003bb76635685359972ea21983224e75876bfa72d1afc675daf"} Jul 20 05:40:50.316099 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:50.315859 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ffe2481232285003bb76635685359972ea21983224e75876bfa72d1afc675daf" Jul 20 05:40:57.484689 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:40:57.484652 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:02.063190 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:02.063155 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87"] Jul 20 05:41:02.066511 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:02.066481 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:02.067693 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:02.067674 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-598bdb-gtk87"] Jul 20 05:41:02.466110 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:02.466029 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:03.653238 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:03.653204 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6046668e-5182-408e-82e8-6096b31c76ea" path="/var/lib/kubelet/pods/6046668e-5182-408e-82e8-6096b31c76ea/volumes" Jul 20 05:41:08.145029 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:08.144991 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:08.212148 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:08.212117 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:08.403299 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:08.403214 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:08.625226 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:08.625194 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:08.977536 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:08.977502 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:09.398775 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:09.397195 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:09.493025 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:09.492992 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:12.406021 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:12.405989 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:12.478895 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:12.478845 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:12.555448 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:12.555408 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:13.072181 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:13.072134 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:13.173016 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:13.172984 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:13.293140 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:13.293110 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:16.905622 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:16.905589 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:17.457548 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:17.457516 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:17.794690 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:17.794656 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:18.412844 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:18.412812 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:20.623424 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:20.623387 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:20.714830 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:20.714775 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:20.786647 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:20.786616 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:21.341251 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:21.341219 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:21.435876 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:21.435838 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:21.501116 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:21.501061 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:25.080368 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:25.080335 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:33.038006 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.037951 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8"] Jul 20 05:41:33.038452 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.038438 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6046668e-5182-408e-82e8-6096b31c76ea" containerName="revoke-keys" Jul 20 05:41:33.038452 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.038453 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6046668e-5182-408e-82e8-6096b31c76ea" containerName="revoke-keys" Jul 20 05:41:33.038536 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.038513 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6046668e-5182-408e-82e8-6096b31c76ea" containerName="revoke-keys" Jul 20 05:41:33.041886 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.041861 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8" Jul 20 05:41:33.044613 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.044418 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:41:33.048725 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.048698 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8"] Jul 20 05:41:33.160674 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.160631 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca724f76-0bdb-44a4-ac5c-4f0c6fce6472-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8\" (UID: \"ca724f76-0bdb-44a4-ac5c-4f0c6fce6472\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8" Jul 20 05:41:33.261257 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.261219 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca724f76-0bdb-44a4-ac5c-4f0c6fce6472-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8\" (UID: \"ca724f76-0bdb-44a4-ac5c-4f0c6fce6472\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8" Jul 20 05:41:33.261821 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.261799 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca724f76-0bdb-44a4-ac5c-4f0c6fce6472-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8\" (UID: \"ca724f76-0bdb-44a4-ac5c-4f0c6fce6472\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8" Jul 20 05:41:33.355686 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.355597 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8" Jul 20 05:41:33.496855 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.496827 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8"] Jul 20 05:41:33.498865 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:41:33.498835 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca724f76_0bdb_44a4_ac5c_4f0c6fce6472.slice/crio-b3e841a39f3a6d5c6541c09f6d045046dec46e5f479a157b420251ff31cc338d WatchSource:0}: Error finding container b3e841a39f3a6d5c6541c09f6d045046dec46e5f479a157b420251ff31cc338d: Status 404 returned error can't find the container with id b3e841a39f3a6d5c6541c09f6d045046dec46e5f479a157b420251ff31cc338d Jul 20 05:41:33.583014 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.582983 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8" event={"ID":"ca724f76-0bdb-44a4-ac5c-4f0c6fce6472","Type":"ContainerStarted","Data":"b3e841a39f3a6d5c6541c09f6d045046dec46e5f479a157b420251ff31cc338d"} Jul 20 05:41:33.656422 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.656347 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:33.774095 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:33.774062 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:34.591053 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:34.590942 2576 generic.go:356] "Generic (PLEG): container finished" podID="ca724f76-0bdb-44a4-ac5c-4f0c6fce6472" containerID="5920d69ea918da85c8ffad2b033ab6f4712e59ad88da277dc8923f518589564e" exitCode=0 Jul 20 05:41:34.591053 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:34.591027 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8" event={"ID":"ca724f76-0bdb-44a4-ac5c-4f0c6fce6472","Type":"ContainerDied","Data":"5920d69ea918da85c8ffad2b033ab6f4712e59ad88da277dc8923f518589564e"} Jul 20 05:41:35.738724 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:35.738701 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8" Jul 20 05:41:35.888512 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:35.888429 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca724f76-0bdb-44a4-ac5c-4f0c6fce6472-maas-api-service-ca\") pod \"ca724f76-0bdb-44a4-ac5c-4f0c6fce6472\" (UID: \"ca724f76-0bdb-44a4-ac5c-4f0c6fce6472\") " Jul 20 05:41:35.888745 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:35.888723 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca724f76-0bdb-44a4-ac5c-4f0c6fce6472-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ca724f76-0bdb-44a4-ac5c-4f0c6fce6472" (UID: "ca724f76-0bdb-44a4-ac5c-4f0c6fce6472"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:41:35.989733 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:35.989702 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca724f76-0bdb-44a4-ac5c-4f0c6fce6472-maas-api-service-ca\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:41:36.605003 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:36.604974 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8" Jul 20 05:41:36.605003 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:36.605002 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8" event={"ID":"ca724f76-0bdb-44a4-ac5c-4f0c6fce6472","Type":"ContainerDied","Data":"b3e841a39f3a6d5c6541c09f6d045046dec46e5f479a157b420251ff31cc338d"} Jul 20 05:41:36.605216 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:36.605047 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3e841a39f3a6d5c6541c09f6d045046dec46e5f479a157b420251ff31cc338d" Jul 20 05:41:42.224267 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:42.224228 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:42.325067 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:42.325030 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:43.605906 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:43.605868 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:43.706643 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:43.706610 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:48.855546 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:48.855514 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:53.114499 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:53.114463 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8"] Jul 20 05:41:53.115263 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:53.115241 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:53.117036 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:53.117014 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce4188-6hfc8"] Jul 20 05:41:53.510695 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:53.510662 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:53.653500 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:53.653460 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca724f76-0bdb-44a4-ac5c-4f0c6fce6472" path="/var/lib/kubelet/pods/ca724f76-0bdb-44a4-ac5c-4f0c6fce6472/volumes" Jul 20 05:41:59.141140 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.141109 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:59.205800 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.205769 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:59.421662 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.421575 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:59.631022 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.630988 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:41:59.753864 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.753829 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml"] Jul 20 05:41:59.754292 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.754279 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca724f76-0bdb-44a4-ac5c-4f0c6fce6472" containerName="revoke-keys" Jul 20 05:41:59.754342 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.754294 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca724f76-0bdb-44a4-ac5c-4f0c6fce6472" containerName="revoke-keys" Jul 20 05:41:59.754394 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.754384 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca724f76-0bdb-44a4-ac5c-4f0c6fce6472" containerName="revoke-keys" Jul 20 05:41:59.757517 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.757498 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml" Jul 20 05:41:59.759790 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.759766 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:41:59.766429 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.766403 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml"] Jul 20 05:41:59.826317 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.826282 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml\" (UID: \"2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml" Jul 20 05:41:59.927169 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.927137 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml\" (UID: \"2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml" Jul 20 05:41:59.927740 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:41:59.927719 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml\" (UID: \"2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml" Jul 20 05:42:00.069619 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:00.069511 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml" Jul 20 05:42:00.211425 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:00.211395 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml"] Jul 20 05:42:00.213900 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:42:00.213872 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f4e3ae2_55b4_498f_bd7a_29c4f6ee3d8f.slice/crio-ff22bab50b0492d5f47e1c8ed15e2aa705a4671e869d3467340c7756a2f1530b WatchSource:0}: Error finding container ff22bab50b0492d5f47e1c8ed15e2aa705a4671e869d3467340c7756a2f1530b: Status 404 returned error can't find the container with id ff22bab50b0492d5f47e1c8ed15e2aa705a4671e869d3467340c7756a2f1530b Jul 20 05:42:00.260183 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:00.260150 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:00.364105 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:00.364028 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:00.754792 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:00.754755 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml" event={"ID":"2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f","Type":"ContainerStarted","Data":"871436bf4efb7aa120a0c2d45bc7c85e2f47faa5db4f84deea178ca7552c9e51"} Jul 20 05:42:00.755016 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:00.754844 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml" event={"ID":"2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f","Type":"ContainerStarted","Data":"ff22bab50b0492d5f47e1c8ed15e2aa705a4671e869d3467340c7756a2f1530b"} Jul 20 05:42:00.769619 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:00.769561 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml" podStartSLOduration=1.769546595 podStartE2EDuration="1.769546595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:41:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:42:00.768340054 +0000 UTC m=+2563.814866983" watchObservedRunningTime="2026-07-20 05:42:00.769546595 +0000 UTC m=+2563.816073501" Jul 20 05:42:01.762243 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:01.762207 2576 generic.go:356] "Generic (PLEG): container finished" podID="2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f" containerID="871436bf4efb7aa120a0c2d45bc7c85e2f47faa5db4f84deea178ca7552c9e51" exitCode=0 Jul 20 05:42:01.762637 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:01.762245 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml" event={"ID":"2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f","Type":"ContainerDied","Data":"871436bf4efb7aa120a0c2d45bc7c85e2f47faa5db4f84deea178ca7552c9e51"} Jul 20 05:42:02.907655 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:02.907630 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml" Jul 20 05:42:02.958194 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:02.958158 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f-maas-api-service-ca\") pod \"2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f\" (UID: \"2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f\") " Jul 20 05:42:02.958500 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:02.958478 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f" (UID: "2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:42:03.060024 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:03.059906 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f-maas-api-service-ca\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:42:03.775653 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:03.775625 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml" Jul 20 05:42:03.775859 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:03.775655 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml" event={"ID":"2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f","Type":"ContainerDied","Data":"ff22bab50b0492d5f47e1c8ed15e2aa705a4671e869d3467340c7756a2f1530b"} Jul 20 05:42:03.775859 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:03.775699 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff22bab50b0492d5f47e1c8ed15e2aa705a4671e869d3467340c7756a2f1530b" Jul 20 05:42:15.240337 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:15.240299 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:19.080907 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:19.080868 2576 scope.go:117] "RemoveContainer" containerID="2501e0e92d9aba85d8860420728bcee9f83419a813ece3a2adc4273eb1b2926f" Jul 20 05:42:19.826824 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:19.826786 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml"] Jul 20 05:42:19.828949 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:19.828918 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:19.829454 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:19.829431 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d0511a-jw5ml"] Jul 20 05:42:20.209052 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:20.208943 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:21.653475 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:21.653445 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f" path="/var/lib/kubelet/pods/2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f/volumes" Jul 20 05:42:25.812723 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:25.812683 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:25.913109 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:25.913078 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:26.079419 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:26.079344 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:26.302761 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:26.302734 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:26.633729 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:26.633698 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:26.938929 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:26.938840 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:27.144854 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:27.144825 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:29.810238 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:29.810203 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:29.900464 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:29.900433 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:29.988128 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:29.988099 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:30.453028 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:30.452988 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:30.612629 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:30.612598 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:30.712764 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:30.712685 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:34.294501 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:34.294465 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:34.614723 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:34.614634 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:34.917598 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:34.917525 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:35.018056 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:35.018022 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:37.903283 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:37.903249 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:38.030234 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:38.030204 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:38.126383 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:38.126350 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:38.227379 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:38.227298 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:38.604617 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:38.604586 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:38.702755 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:38.702696 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:40.817035 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:40.816928 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:40.914970 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:40.914936 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:42.320911 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:42.320882 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:51.363538 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:51.363497 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:51.464339 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:51.464307 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:54.647557 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:54.647516 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:59.325593 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:59.325559 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:42:59.426075 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:42:59.426041 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:43:19.104974 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:43:19.104937 2576 scope.go:117] "RemoveContainer" containerID="130af06b015c0a714752589b374ac33109243e970579c026948bf5a46f1dd28e" Jul 20 05:43:23.861522 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:43:23.861488 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:44:17.975321 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:44:17.975247 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:44:17.979532 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:44:17.979508 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:44:17.980689 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:44:17.980664 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:44:17.984795 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:44:17.984769 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:44:23.230257 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:44:23.230219 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:44:23.284093 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:44:23.284065 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:44:24.166744 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:44:24.166715 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:44:24.266722 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:44:24.266693 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:45:00.130782 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.130750 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742105-jclv4"] Jul 20 05:45:00.131253 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.131192 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f" containerName="revoke-keys" Jul 20 05:45:00.131253 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.131203 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f" containerName="revoke-keys" Jul 20 05:45:00.131322 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.131283 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f4e3ae2-55b4-498f-bd7a-29c4f6ee3d8f" containerName="revoke-keys" Jul 20 05:45:00.134355 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.134334 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742105-jclv4" Jul 20 05:45:00.136699 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.136674 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:45:00.137383 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.137358 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82cedd-29742105-f9x7p"] Jul 20 05:45:00.141377 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.141355 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m"] Jul 20 05:45:00.141496 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.141463 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82cedd-29742105-f9x7p" Jul 20 05:45:00.146405 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.146385 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742105-jclv4"] Jul 20 05:45:00.146620 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.146604 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m" Jul 20 05:45:00.149336 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.149312 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82cedd-29742105-f9x7p"] Jul 20 05:45:00.149738 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.149717 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742105-jclv4" Jul 20 05:45:00.158106 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.158083 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m"] Jul 20 05:45:00.160548 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.160530 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82cedd-29742105-f9x7p" Jul 20 05:45:00.196340 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.196319 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m" Jul 20 05:45:00.321149 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.320228 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742105-jclv4"] Jul 20 05:45:00.381266 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:45:00.381195 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f3f840d_6351_44ec_90f8_4019d66628e0.slice/crio-cf2a13e9e460f8314294eba5ec9c32415f89a2bf859e2fef1cb5ae9d5abeb886 WatchSource:0}: Error finding container cf2a13e9e460f8314294eba5ec9c32415f89a2bf859e2fef1cb5ae9d5abeb886: Status 404 returned error can't find the container with id cf2a13e9e460f8314294eba5ec9c32415f89a2bf859e2fef1cb5ae9d5abeb886 Jul 20 05:45:00.382152 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.382127 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m"] Jul 20 05:45:00.549662 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.549630 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82cedd-29742105-f9x7p"] Jul 20 05:45:00.550812 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:45:00.550782 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c19e84d_010e_46e5_be24_8ac740f921bc.slice/crio-14744782bdcc4e2ec88ed50160298bece72c6e941b711a2b15aa0bd2512b1eb7 WatchSource:0}: Error finding container 14744782bdcc4e2ec88ed50160298bece72c6e941b711a2b15aa0bd2512b1eb7: Status 404 returned error can't find the container with id 14744782bdcc4e2ec88ed50160298bece72c6e941b711a2b15aa0bd2512b1eb7 Jul 20 05:45:00.834847 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.834764 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82cedd-29742105-f9x7p" event={"ID":"8c19e84d-010e-46e5-be24-8ac740f921bc","Type":"ContainerStarted","Data":"14744782bdcc4e2ec88ed50160298bece72c6e941b711a2b15aa0bd2512b1eb7"} Jul 20 05:45:00.836377 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.836352 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742105-jclv4" event={"ID":"899a75a2-63c1-4423-92c7-63b236c82dde","Type":"ContainerStarted","Data":"c09541309d2133fc96d58069c081ebecaa3f575eaf75537c54f1babbde673529"} Jul 20 05:45:00.836505 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.836400 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742105-jclv4" event={"ID":"899a75a2-63c1-4423-92c7-63b236c82dde","Type":"ContainerStarted","Data":"87eea96efb056879029167a37b45ae34185e2d976e937be3cc11bbf57279bc41"} Jul 20 05:45:00.838215 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.838190 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m" event={"ID":"5f3f840d-6351-44ec-90f8-4019d66628e0","Type":"ContainerStarted","Data":"4a25c94960dcf74feda446b5156ad6daa747f5e8a0af154b7a9ae9b26f6d4580"} Jul 20 05:45:00.838318 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.838227 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m" event={"ID":"5f3f840d-6351-44ec-90f8-4019d66628e0","Type":"ContainerStarted","Data":"cf2a13e9e460f8314294eba5ec9c32415f89a2bf859e2fef1cb5ae9d5abeb886"} Jul 20 05:45:00.852256 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.852217 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742105-jclv4" podStartSLOduration=0.852203818 podStartE2EDuration="852.203818ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:45:00.851118273 +0000 UTC m=+2743.897645178" watchObservedRunningTime="2026-07-20 05:45:00.852203818 +0000 UTC m=+2743.898730722" Jul 20 05:45:00.865253 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:00.865214 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m" podStartSLOduration=0.865202421 podStartE2EDuration="865.202421ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:45:00.863875256 +0000 UTC m=+2743.910402202" watchObservedRunningTime="2026-07-20 05:45:00.865202421 +0000 UTC m=+2743.911729327" Jul 20 05:45:01.687949 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:01.687865 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:45:01.845757 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:01.845721 2576 generic.go:356] "Generic (PLEG): container finished" podID="5f3f840d-6351-44ec-90f8-4019d66628e0" containerID="4a25c94960dcf74feda446b5156ad6daa747f5e8a0af154b7a9ae9b26f6d4580" exitCode=0 Jul 20 05:45:01.845992 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:01.845816 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m" event={"ID":"5f3f840d-6351-44ec-90f8-4019d66628e0","Type":"ContainerDied","Data":"4a25c94960dcf74feda446b5156ad6daa747f5e8a0af154b7a9ae9b26f6d4580"} Jul 20 05:45:01.847335 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:01.847314 2576 generic.go:356] "Generic (PLEG): container finished" podID="8c19e84d-010e-46e5-be24-8ac740f921bc" containerID="432207f02f2469476c91a9d117a113e4a1647370af54605fce682d71f096c87f" exitCode=0 Jul 20 05:45:01.847431 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:01.847371 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82cedd-29742105-f9x7p" event={"ID":"8c19e84d-010e-46e5-be24-8ac740f921bc","Type":"ContainerDied","Data":"432207f02f2469476c91a9d117a113e4a1647370af54605fce682d71f096c87f"} Jul 20 05:45:01.848805 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:01.848777 2576 generic.go:356] "Generic (PLEG): container finished" podID="899a75a2-63c1-4423-92c7-63b236c82dde" containerID="c09541309d2133fc96d58069c081ebecaa3f575eaf75537c54f1babbde673529" exitCode=0 Jul 20 05:45:01.848899 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:01.848872 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742105-jclv4" event={"ID":"899a75a2-63c1-4423-92c7-63b236c82dde","Type":"ContainerDied","Data":"c09541309d2133fc96d58069c081ebecaa3f575eaf75537c54f1babbde673529"} Jul 20 05:45:02.998242 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:02.998215 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82cedd-29742105-f9x7p" Jul 20 05:45:03.043147 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:03.043123 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742105-jclv4" Jul 20 05:45:03.046311 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:03.046288 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m" Jul 20 05:45:03.862319 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:03.862294 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82cedd-29742105-f9x7p" Jul 20 05:45:03.862552 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:03.862294 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82cedd-29742105-f9x7p" event={"ID":"8c19e84d-010e-46e5-be24-8ac740f921bc","Type":"ContainerDied","Data":"14744782bdcc4e2ec88ed50160298bece72c6e941b711a2b15aa0bd2512b1eb7"} Jul 20 05:45:03.862552 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:03.862432 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14744782bdcc4e2ec88ed50160298bece72c6e941b711a2b15aa0bd2512b1eb7" Jul 20 05:45:03.863859 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:03.863836 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742105-jclv4" Jul 20 05:45:03.864005 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:03.863829 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742105-jclv4" event={"ID":"899a75a2-63c1-4423-92c7-63b236c82dde","Type":"ContainerDied","Data":"87eea96efb056879029167a37b45ae34185e2d976e937be3cc11bbf57279bc41"} Jul 20 05:45:03.864005 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:03.863992 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87eea96efb056879029167a37b45ae34185e2d976e937be3cc11bbf57279bc41" Jul 20 05:45:03.865459 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:03.865439 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m" Jul 20 05:45:03.865555 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:03.865478 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m" event={"ID":"5f3f840d-6351-44ec-90f8-4019d66628e0","Type":"ContainerDied","Data":"cf2a13e9e460f8314294eba5ec9c32415f89a2bf859e2fef1cb5ae9d5abeb886"} Jul 20 05:45:03.865555 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:03.865520 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf2a13e9e460f8314294eba5ec9c32415f89a2bf859e2fef1cb5ae9d5abeb886" Jul 20 05:45:16.920533 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:16.920500 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:45:30.736712 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:30.736681 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:45:31.618528 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:31.618497 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:45:47.651869 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:45:47.651771 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:46:30.120084 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:46:30.120035 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:46:32.608006 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:46:32.607971 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:46:32.709127 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:46:32.709094 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:47:06.182326 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:47:06.182289 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:47:07.230187 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:47:07.230153 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:47:19.154308 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:47:19.154236 2576 scope.go:117] "RemoveContainer" containerID="26e57db18a0dd1abb9b825a28b647d195192d8bb55962b0d64d988734dcf032f" Jul 20 05:47:19.163447 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:47:19.163427 2576 scope.go:117] "RemoveContainer" containerID="63dd08cdaf32ec8f0aa9c6e8cffe39aa53b4d53dd95ce38f43859a5f1bdef057" Jul 20 05:47:22.558394 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:47:22.558359 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:47:36.746376 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:47:36.746342 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:47:41.606633 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:47:41.606590 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:47:41.706776 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:47:41.706739 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:47:52.170616 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:47:52.170574 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:47:52.272201 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:47:52.272165 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:00.182381 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:00.182346 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:00.283422 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:00.283388 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:16.624728 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:16.624695 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:17.554611 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:17.554575 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:18.531415 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.531381 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7760-139936997499440"] Jul 20 05:48:18.531796 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.531783 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c19e84d-010e-46e5-be24-8ac740f921bc" containerName="cleanup" Jul 20 05:48:18.531840 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.531797 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c19e84d-010e-46e5-be24-8ac740f921bc" containerName="cleanup" Jul 20 05:48:18.531840 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.531808 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f3f840d-6351-44ec-90f8-4019d66628e0" containerName="cleanup" Jul 20 05:48:18.531840 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.531814 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f3f840d-6351-44ec-90f8-4019d66628e0" containerName="cleanup" Jul 20 05:48:18.531840 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.531820 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="899a75a2-63c1-4423-92c7-63b236c82dde" containerName="cleanup" Jul 20 05:48:18.531840 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.531826 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="899a75a2-63c1-4423-92c7-63b236c82dde" containerName="cleanup" Jul 20 05:48:18.532018 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.531906 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c19e84d-010e-46e5-be24-8ac740f921bc" containerName="cleanup" Jul 20 05:48:18.532018 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.531913 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="899a75a2-63c1-4423-92c7-63b236c82dde" containerName="cleanup" Jul 20 05:48:18.532018 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.531920 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f3f840d-6351-44ec-90f8-4019d66628e0" containerName="cleanup" Jul 20 05:48:18.535148 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.535129 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997499440" Jul 20 05:48:18.538347 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.538321 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-jkw8j\"" Jul 20 05:48:18.542108 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.542083 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7760-139936997499440"] Jul 20 05:48:18.659292 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.659257 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qswd7\" (UniqueName: \"kubernetes.io/projected/f2dd3071-8db8-4911-8e7f-9ec0a880b654-kube-api-access-qswd7\") pod \"test-curl-7760-139936997499440\" (UID: \"f2dd3071-8db8-4911-8e7f-9ec0a880b654\") " pod="opendatahub/test-curl-7760-139936997499440" Jul 20 05:48:18.761118 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.761086 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qswd7\" (UniqueName: \"kubernetes.io/projected/f2dd3071-8db8-4911-8e7f-9ec0a880b654-kube-api-access-qswd7\") pod \"test-curl-7760-139936997499440\" (UID: \"f2dd3071-8db8-4911-8e7f-9ec0a880b654\") " pod="opendatahub/test-curl-7760-139936997499440" Jul 20 05:48:18.768719 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.768694 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qswd7\" (UniqueName: \"kubernetes.io/projected/f2dd3071-8db8-4911-8e7f-9ec0a880b654-kube-api-access-qswd7\") pod \"test-curl-7760-139936997499440\" (UID: \"f2dd3071-8db8-4911-8e7f-9ec0a880b654\") " pod="opendatahub/test-curl-7760-139936997499440" Jul 20 05:48:18.845271 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.845177 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997499440" Jul 20 05:48:18.983061 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.983036 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7760-139936997499440"] Jul 20 05:48:18.984934 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:48:18.984905 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2dd3071_8db8_4911_8e7f_9ec0a880b654.slice/crio-ae0406cc59de789c3afbb6e7e307ce4ae1f15049474ef0dc8a4eb37f731fd86a WatchSource:0}: Error finding container ae0406cc59de789c3afbb6e7e307ce4ae1f15049474ef0dc8a4eb37f731fd86a: Status 404 returned error can't find the container with id ae0406cc59de789c3afbb6e7e307ce4ae1f15049474ef0dc8a4eb37f731fd86a Jul 20 05:48:18.986292 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:18.986275 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 05:48:19.042716 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:19.042688 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7760-139936997499440" event={"ID":"f2dd3071-8db8-4911-8e7f-9ec0a880b654","Type":"ContainerStarted","Data":"ae0406cc59de789c3afbb6e7e307ce4ae1f15049474ef0dc8a4eb37f731fd86a"} Jul 20 05:48:19.189007 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:19.188917 2576 scope.go:117] "RemoveContainer" containerID="871436bf4efb7aa120a0c2d45bc7c85e2f47faa5db4f84deea178ca7552c9e51" Jul 20 05:48:19.197922 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:19.197905 2576 scope.go:117] "RemoveContainer" containerID="5920d69ea918da85c8ffad2b033ab6f4712e59ad88da277dc8923f518589564e" Jul 20 05:48:20.049729 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:20.049697 2576 generic.go:356] "Generic (PLEG): container finished" podID="f2dd3071-8db8-4911-8e7f-9ec0a880b654" containerID="139746d491a141bd9338b1b12562643a1af5f082933ad95276a31cd880d64657" exitCode=0 Jul 20 05:48:20.050183 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:20.049776 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7760-139936997499440" event={"ID":"f2dd3071-8db8-4911-8e7f-9ec0a880b654","Type":"ContainerDied","Data":"139746d491a141bd9338b1b12562643a1af5f082933ad95276a31cd880d64657"} Jul 20 05:48:21.197114 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.197090 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997499440" Jul 20 05:48:21.216344 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.216317 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7760-139936997499440_f2dd3071-8db8-4911-8e7f-9ec0a880b654/test-curl-7760-139936997499440/0.log" Jul 20 05:48:21.240274 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.240230 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7760-139936997499440"] Jul 20 05:48:21.243453 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.243430 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7760-139936997499440"] Jul 20 05:48:21.286525 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.286505 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qswd7\" (UniqueName: \"kubernetes.io/projected/f2dd3071-8db8-4911-8e7f-9ec0a880b654-kube-api-access-qswd7\") pod \"f2dd3071-8db8-4911-8e7f-9ec0a880b654\" (UID: \"f2dd3071-8db8-4911-8e7f-9ec0a880b654\") " Jul 20 05:48:21.288538 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.288511 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2dd3071-8db8-4911-8e7f-9ec0a880b654-kube-api-access-qswd7" (OuterVolumeSpecName: "kube-api-access-qswd7") pod "f2dd3071-8db8-4911-8e7f-9ec0a880b654" (UID: "f2dd3071-8db8-4911-8e7f-9ec0a880b654"). InnerVolumeSpecName "kube-api-access-qswd7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:48:21.370266 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.370197 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7760-139936997597488"] Jul 20 05:48:21.370645 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.370632 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2dd3071-8db8-4911-8e7f-9ec0a880b654" containerName="test-curl-7760-139936997499440" Jul 20 05:48:21.370691 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.370647 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2dd3071-8db8-4911-8e7f-9ec0a880b654" containerName="test-curl-7760-139936997499440" Jul 20 05:48:21.370740 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.370729 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2dd3071-8db8-4911-8e7f-9ec0a880b654" containerName="test-curl-7760-139936997499440" Jul 20 05:48:21.373916 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.373892 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997597488" Jul 20 05:48:21.377670 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.377645 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7760-139936997597488"] Jul 20 05:48:21.387406 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.387383 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qswd7\" (UniqueName: \"kubernetes.io/projected/f2dd3071-8db8-4911-8e7f-9ec0a880b654-kube-api-access-qswd7\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:48:21.488666 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.488638 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nhhb\" (UniqueName: \"kubernetes.io/projected/554b8043-6665-4a11-80d2-97d7c5325138-kube-api-access-4nhhb\") pod \"test-curl-7760-139936997597488\" (UID: \"554b8043-6665-4a11-80d2-97d7c5325138\") " pod="opendatahub/test-curl-7760-139936997597488" Jul 20 05:48:21.589921 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.589889 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nhhb\" (UniqueName: \"kubernetes.io/projected/554b8043-6665-4a11-80d2-97d7c5325138-kube-api-access-4nhhb\") pod \"test-curl-7760-139936997597488\" (UID: \"554b8043-6665-4a11-80d2-97d7c5325138\") " pod="opendatahub/test-curl-7760-139936997597488" Jul 20 05:48:21.598073 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.598048 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nhhb\" (UniqueName: \"kubernetes.io/projected/554b8043-6665-4a11-80d2-97d7c5325138-kube-api-access-4nhhb\") pod \"test-curl-7760-139936997597488\" (UID: \"554b8043-6665-4a11-80d2-97d7c5325138\") " pod="opendatahub/test-curl-7760-139936997597488" Jul 20 05:48:21.653921 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.653846 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2dd3071-8db8-4911-8e7f-9ec0a880b654" path="/var/lib/kubelet/pods/f2dd3071-8db8-4911-8e7f-9ec0a880b654/volumes" Jul 20 05:48:21.684483 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.684460 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997597488" Jul 20 05:48:21.817676 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:21.817652 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7760-139936997597488"] Jul 20 05:48:21.819305 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:48:21.819279 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod554b8043_6665_4a11_80d2_97d7c5325138.slice/crio-6dc8e0f1facdbc13e73a1c159186775b27cf15efdcb3b5ff906857cf5ac03444 WatchSource:0}: Error finding container 6dc8e0f1facdbc13e73a1c159186775b27cf15efdcb3b5ff906857cf5ac03444: Status 404 returned error can't find the container with id 6dc8e0f1facdbc13e73a1c159186775b27cf15efdcb3b5ff906857cf5ac03444 Jul 20 05:48:22.063178 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:22.063144 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7760-139936997597488" event={"ID":"554b8043-6665-4a11-80d2-97d7c5325138","Type":"ContainerStarted","Data":"6dc8e0f1facdbc13e73a1c159186775b27cf15efdcb3b5ff906857cf5ac03444"} Jul 20 05:48:22.064416 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:22.064394 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997499440" Jul 20 05:48:22.064416 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:22.064413 2576 scope.go:117] "RemoveContainer" containerID="139746d491a141bd9338b1b12562643a1af5f082933ad95276a31cd880d64657" Jul 20 05:48:23.073238 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:23.073201 2576 generic.go:356] "Generic (PLEG): container finished" podID="554b8043-6665-4a11-80d2-97d7c5325138" containerID="912acdfe27e20a420acf40e088b8ad5f22a5f1f93eee3d013001e10495144155" exitCode=0 Jul 20 05:48:23.073624 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:23.073257 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7760-139936997597488" event={"ID":"554b8043-6665-4a11-80d2-97d7c5325138","Type":"ContainerDied","Data":"912acdfe27e20a420acf40e088b8ad5f22a5f1f93eee3d013001e10495144155"} Jul 20 05:48:24.219391 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.219367 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997597488" Jul 20 05:48:24.242796 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.242774 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7760-139936997597488_554b8043-6665-4a11-80d2-97d7c5325138/test-curl-7760-139936997597488/0.log" Jul 20 05:48:24.272549 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.272522 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7760-139936997597488"] Jul 20 05:48:24.275397 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.275373 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7760-139936997597488"] Jul 20 05:48:24.318420 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.318397 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nhhb\" (UniqueName: \"kubernetes.io/projected/554b8043-6665-4a11-80d2-97d7c5325138-kube-api-access-4nhhb\") pod \"554b8043-6665-4a11-80d2-97d7c5325138\" (UID: \"554b8043-6665-4a11-80d2-97d7c5325138\") " Jul 20 05:48:24.320349 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.320327 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/554b8043-6665-4a11-80d2-97d7c5325138-kube-api-access-4nhhb" (OuterVolumeSpecName: "kube-api-access-4nhhb") pod "554b8043-6665-4a11-80d2-97d7c5325138" (UID: "554b8043-6665-4a11-80d2-97d7c5325138"). InnerVolumeSpecName "kube-api-access-4nhhb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:48:24.419502 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.419437 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nhhb\" (UniqueName: \"kubernetes.io/projected/554b8043-6665-4a11-80d2-97d7c5325138-kube-api-access-4nhhb\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:48:24.438128 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.438100 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7760-139936997540176"] Jul 20 05:48:24.438564 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.438549 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="554b8043-6665-4a11-80d2-97d7c5325138" containerName="test-curl-7760-139936997597488" Jul 20 05:48:24.438607 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.438566 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="554b8043-6665-4a11-80d2-97d7c5325138" containerName="test-curl-7760-139936997597488" Jul 20 05:48:24.438655 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.438646 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="554b8043-6665-4a11-80d2-97d7c5325138" containerName="test-curl-7760-139936997597488" Jul 20 05:48:24.441798 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.441783 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:24.449516 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.449452 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7760-139936997540176"] Jul 20 05:48:24.620666 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.620625 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw8s8\" (UniqueName: \"kubernetes.io/projected/2bd6efcf-01e0-432f-9392-823b3ea60482-kube-api-access-dw8s8\") pod \"test-curl-7760-139936997540176\" (UID: \"2bd6efcf-01e0-432f-9392-823b3ea60482\") " pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:24.722000 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.721893 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw8s8\" (UniqueName: \"kubernetes.io/projected/2bd6efcf-01e0-432f-9392-823b3ea60482-kube-api-access-dw8s8\") pod \"test-curl-7760-139936997540176\" (UID: \"2bd6efcf-01e0-432f-9392-823b3ea60482\") " pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:24.730586 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.730556 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw8s8\" (UniqueName: \"kubernetes.io/projected/2bd6efcf-01e0-432f-9392-823b3ea60482-kube-api-access-dw8s8\") pod \"test-curl-7760-139936997540176\" (UID: \"2bd6efcf-01e0-432f-9392-823b3ea60482\") " pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:24.762623 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.762587 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:24.901127 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:24.901100 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7760-139936997540176"] Jul 20 05:48:24.901933 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:48:24.901911 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bd6efcf_01e0_432f_9392_823b3ea60482.slice/crio-35e590715c67a1578b6f20cea0f88256ef81b42fb9be7be11f2de9d42b824c1a WatchSource:0}: Error finding container 35e590715c67a1578b6f20cea0f88256ef81b42fb9be7be11f2de9d42b824c1a: Status 404 returned error can't find the container with id 35e590715c67a1578b6f20cea0f88256ef81b42fb9be7be11f2de9d42b824c1a Jul 20 05:48:25.087668 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:25.087640 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997597488" Jul 20 05:48:25.087852 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:25.087673 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6dc8e0f1facdbc13e73a1c159186775b27cf15efdcb3b5ff906857cf5ac03444" Jul 20 05:48:25.088926 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:25.088903 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7760-139936997540176" event={"ID":"2bd6efcf-01e0-432f-9392-823b3ea60482","Type":"ContainerStarted","Data":"35e590715c67a1578b6f20cea0f88256ef81b42fb9be7be11f2de9d42b824c1a"} Jul 20 05:48:25.098628 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:48:25.098603 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7760-139936997597488\" is forbidden: User \"system:node:ip-10-0-128-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-33.ec2.internal' and this object" podUID="554b8043-6665-4a11-80d2-97d7c5325138" pod="opendatahub/test-curl-7760-139936997597488" Jul 20 05:48:25.655577 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:25.655543 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="554b8043-6665-4a11-80d2-97d7c5325138" path="/var/lib/kubelet/pods/554b8043-6665-4a11-80d2-97d7c5325138/volumes" Jul 20 05:48:26.097191 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:26.097157 2576 generic.go:356] "Generic (PLEG): container finished" podID="2bd6efcf-01e0-432f-9392-823b3ea60482" containerID="c8fca335825e523c61cee583f90be774366412ce7d2af9d229896fad31f687d7" exitCode=0 Jul 20 05:48:26.097360 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:26.097236 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7760-139936997540176" event={"ID":"2bd6efcf-01e0-432f-9392-823b3ea60482","Type":"ContainerDied","Data":"c8fca335825e523c61cee583f90be774366412ce7d2af9d229896fad31f687d7"} Jul 20 05:48:27.240868 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.240844 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:27.260752 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.260731 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7760-139936997540176_2bd6efcf-01e0-432f-9392-823b3ea60482/test-curl-7760-139936997540176/0.log" Jul 20 05:48:27.284598 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.284575 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7760-139936997540176"] Jul 20 05:48:27.289686 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.289663 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7760-139936997540176"] Jul 20 05:48:27.349223 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.349195 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dw8s8\" (UniqueName: \"kubernetes.io/projected/2bd6efcf-01e0-432f-9392-823b3ea60482-kube-api-access-dw8s8\") pod \"2bd6efcf-01e0-432f-9392-823b3ea60482\" (UID: \"2bd6efcf-01e0-432f-9392-823b3ea60482\") " Jul 20 05:48:27.351213 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.351184 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bd6efcf-01e0-432f-9392-823b3ea60482-kube-api-access-dw8s8" (OuterVolumeSpecName: "kube-api-access-dw8s8") pod "2bd6efcf-01e0-432f-9392-823b3ea60482" (UID: "2bd6efcf-01e0-432f-9392-823b3ea60482"). InnerVolumeSpecName "kube-api-access-dw8s8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:48:27.414910 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.414881 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7760-139936997540176"] Jul 20 05:48:27.415304 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.415291 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bd6efcf-01e0-432f-9392-823b3ea60482" containerName="test-curl-7760-139936997540176" Jul 20 05:48:27.415347 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.415306 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bd6efcf-01e0-432f-9392-823b3ea60482" containerName="test-curl-7760-139936997540176" Jul 20 05:48:27.415402 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.415393 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bd6efcf-01e0-432f-9392-823b3ea60482" containerName="test-curl-7760-139936997540176" Jul 20 05:48:27.418507 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.418492 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:27.425689 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.425664 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7760-139936997540176"] Jul 20 05:48:27.450820 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.450795 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw8s8\" (UniqueName: \"kubernetes.io/projected/2bd6efcf-01e0-432f-9392-823b3ea60482-kube-api-access-dw8s8\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:48:27.551750 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.551721 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94nfm\" (UniqueName: \"kubernetes.io/projected/2a2da6aa-9a39-46de-8243-9b04e828a3a3-kube-api-access-94nfm\") pod \"test-curl-7760-139936997540176\" (UID: \"2a2da6aa-9a39-46de-8243-9b04e828a3a3\") " pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:27.652683 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.652610 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94nfm\" (UniqueName: \"kubernetes.io/projected/2a2da6aa-9a39-46de-8243-9b04e828a3a3-kube-api-access-94nfm\") pod \"test-curl-7760-139936997540176\" (UID: \"2a2da6aa-9a39-46de-8243-9b04e828a3a3\") " pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:27.654151 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.654130 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bd6efcf-01e0-432f-9392-823b3ea60482" path="/var/lib/kubelet/pods/2bd6efcf-01e0-432f-9392-823b3ea60482/volumes" Jul 20 05:48:27.660438 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.660417 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94nfm\" (UniqueName: \"kubernetes.io/projected/2a2da6aa-9a39-46de-8243-9b04e828a3a3-kube-api-access-94nfm\") pod \"test-curl-7760-139936997540176\" (UID: \"2a2da6aa-9a39-46de-8243-9b04e828a3a3\") " pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:27.729464 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.729438 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:27.861176 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:27.861145 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7760-139936997540176"] Jul 20 05:48:28.110314 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:28.110277 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7760-139936997540176" event={"ID":"2a2da6aa-9a39-46de-8243-9b04e828a3a3","Type":"ContainerStarted","Data":"e254fda5aeec170d095291ea090819cdf77d0e3b9004687c743d6e2da25bd4ec"} Jul 20 05:48:28.111503 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:28.111480 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:28.111612 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:28.111485 2576 scope.go:117] "RemoveContainer" containerID="c8fca335825e523c61cee583f90be774366412ce7d2af9d229896fad31f687d7" Jul 20 05:48:29.119494 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:29.119459 2576 generic.go:356] "Generic (PLEG): container finished" podID="2a2da6aa-9a39-46de-8243-9b04e828a3a3" containerID="dd23942d64fd14edc060a168d6fcf0d83ed7bb220d3d36c2bf5235d0a5c97a1d" exitCode=0 Jul 20 05:48:29.119901 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:29.119540 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7760-139936997540176" event={"ID":"2a2da6aa-9a39-46de-8243-9b04e828a3a3","Type":"ContainerDied","Data":"dd23942d64fd14edc060a168d6fcf0d83ed7bb220d3d36c2bf5235d0a5c97a1d"} Jul 20 05:48:30.266230 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.266205 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:30.280510 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.280314 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94nfm\" (UniqueName: \"kubernetes.io/projected/2a2da6aa-9a39-46de-8243-9b04e828a3a3-kube-api-access-94nfm\") pod \"2a2da6aa-9a39-46de-8243-9b04e828a3a3\" (UID: \"2a2da6aa-9a39-46de-8243-9b04e828a3a3\") " Jul 20 05:48:30.282875 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.282849 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a2da6aa-9a39-46de-8243-9b04e828a3a3-kube-api-access-94nfm" (OuterVolumeSpecName: "kube-api-access-94nfm") pod "2a2da6aa-9a39-46de-8243-9b04e828a3a3" (UID: "2a2da6aa-9a39-46de-8243-9b04e828a3a3"). InnerVolumeSpecName "kube-api-access-94nfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:48:30.286411 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.286368 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7760-139936997540176_2a2da6aa-9a39-46de-8243-9b04e828a3a3/test-curl-7760-139936997540176/0.log" Jul 20 05:48:30.311601 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.311575 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7760-139936997540176"] Jul 20 05:48:30.315283 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.315263 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7760-139936997540176"] Jul 20 05:48:30.381475 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.381398 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94nfm\" (UniqueName: \"kubernetes.io/projected/2a2da6aa-9a39-46de-8243-9b04e828a3a3-kube-api-access-94nfm\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:48:30.440247 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.440206 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7760-139936997725216"] Jul 20 05:48:30.440699 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.440681 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a2da6aa-9a39-46de-8243-9b04e828a3a3" containerName="test-curl-7760-139936997540176" Jul 20 05:48:30.440790 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.440701 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a2da6aa-9a39-46de-8243-9b04e828a3a3" containerName="test-curl-7760-139936997540176" Jul 20 05:48:30.440847 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.440825 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a2da6aa-9a39-46de-8243-9b04e828a3a3" containerName="test-curl-7760-139936997540176" Jul 20 05:48:30.443873 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.443854 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:30.450509 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.450462 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7760-139936997725216"] Jul 20 05:48:30.481863 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.481833 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh98l\" (UniqueName: \"kubernetes.io/projected/76b16151-2053-4941-833b-7c0748667372-kube-api-access-kh98l\") pod \"test-curl-7760-139936997725216\" (UID: \"76b16151-2053-4941-833b-7c0748667372\") " pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:30.583059 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.583025 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh98l\" (UniqueName: \"kubernetes.io/projected/76b16151-2053-4941-833b-7c0748667372-kube-api-access-kh98l\") pod \"test-curl-7760-139936997725216\" (UID: \"76b16151-2053-4941-833b-7c0748667372\") " pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:30.591280 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.591250 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh98l\" (UniqueName: \"kubernetes.io/projected/76b16151-2053-4941-833b-7c0748667372-kube-api-access-kh98l\") pod \"test-curl-7760-139936997725216\" (UID: \"76b16151-2053-4941-833b-7c0748667372\") " pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:30.754124 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.754089 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:30.887701 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:30.887669 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7760-139936997725216"] Jul 20 05:48:30.888888 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:48:30.888860 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76b16151_2053_4941_833b_7c0748667372.slice/crio-fa46a2fba5fe3b09c05d16e26ab2cc469cc1e6fc8d40fc241932539142d61d75 WatchSource:0}: Error finding container fa46a2fba5fe3b09c05d16e26ab2cc469cc1e6fc8d40fc241932539142d61d75: Status 404 returned error can't find the container with id fa46a2fba5fe3b09c05d16e26ab2cc469cc1e6fc8d40fc241932539142d61d75 Jul 20 05:48:31.131862 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:31.131769 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7760-139936997725216" event={"ID":"76b16151-2053-4941-833b-7c0748667372","Type":"ContainerStarted","Data":"fa46a2fba5fe3b09c05d16e26ab2cc469cc1e6fc8d40fc241932539142d61d75"} Jul 20 05:48:31.133083 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:31.133061 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:31.133083 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:31.133078 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e254fda5aeec170d095291ea090819cdf77d0e3b9004687c743d6e2da25bd4ec" Jul 20 05:48:31.144445 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:48:31.144414 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7760-139936997540176\" is forbidden: User \"system:node:ip-10-0-128-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-33.ec2.internal' and this object" podUID="2a2da6aa-9a39-46de-8243-9b04e828a3a3" pod="opendatahub/test-curl-7760-139936997540176" Jul 20 05:48:31.654190 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:31.654159 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a2da6aa-9a39-46de-8243-9b04e828a3a3" path="/var/lib/kubelet/pods/2a2da6aa-9a39-46de-8243-9b04e828a3a3/volumes" Jul 20 05:48:32.140088 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:32.140051 2576 generic.go:356] "Generic (PLEG): container finished" podID="76b16151-2053-4941-833b-7c0748667372" containerID="ccfdb840c19b1a0474c84ca49a00d027e4f4b835acbe9640c996c6b5bd625f0b" exitCode=0 Jul 20 05:48:32.140270 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:32.140141 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7760-139936997725216" event={"ID":"76b16151-2053-4941-833b-7c0748667372","Type":"ContainerDied","Data":"ccfdb840c19b1a0474c84ca49a00d027e4f4b835acbe9640c996c6b5bd625f0b"} Jul 20 05:48:33.288199 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.288174 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:33.308049 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.308020 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kh98l\" (UniqueName: \"kubernetes.io/projected/76b16151-2053-4941-833b-7c0748667372-kube-api-access-kh98l\") pod \"76b16151-2053-4941-833b-7c0748667372\" (UID: \"76b16151-2053-4941-833b-7c0748667372\") " Jul 20 05:48:33.310425 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.310399 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76b16151-2053-4941-833b-7c0748667372-kube-api-access-kh98l" (OuterVolumeSpecName: "kube-api-access-kh98l") pod "76b16151-2053-4941-833b-7c0748667372" (UID: "76b16151-2053-4941-833b-7c0748667372"). InnerVolumeSpecName "kube-api-access-kh98l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:48:33.313888 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.313864 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7760-139936997725216_76b16151-2053-4941-833b-7c0748667372/test-curl-7760-139936997725216/0.log" Jul 20 05:48:33.341408 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.341381 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7760-139936997725216"] Jul 20 05:48:33.346971 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.346933 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7760-139936997725216"] Jul 20 05:48:33.409163 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.409086 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kh98l\" (UniqueName: \"kubernetes.io/projected/76b16151-2053-4941-833b-7c0748667372-kube-api-access-kh98l\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:48:33.478964 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.478926 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7760-139936997725216"] Jul 20 05:48:33.479396 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.479382 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76b16151-2053-4941-833b-7c0748667372" containerName="test-curl-7760-139936997725216" Jul 20 05:48:33.479439 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.479398 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="76b16151-2053-4941-833b-7c0748667372" containerName="test-curl-7760-139936997725216" Jul 20 05:48:33.479472 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.479462 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="76b16151-2053-4941-833b-7c0748667372" containerName="test-curl-7760-139936997725216" Jul 20 05:48:33.482549 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.482533 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:33.489691 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.489670 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7760-139936997725216"] Jul 20 05:48:33.509952 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.509923 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcx4j\" (UniqueName: \"kubernetes.io/projected/2c7423c0-ed5e-4524-b955-b050a3bf286c-kube-api-access-rcx4j\") pod \"test-curl-7760-139936997725216\" (UID: \"2c7423c0-ed5e-4524-b955-b050a3bf286c\") " pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:33.611349 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.611323 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcx4j\" (UniqueName: \"kubernetes.io/projected/2c7423c0-ed5e-4524-b955-b050a3bf286c-kube-api-access-rcx4j\") pod \"test-curl-7760-139936997725216\" (UID: \"2c7423c0-ed5e-4524-b955-b050a3bf286c\") " pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:33.618840 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.618816 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcx4j\" (UniqueName: \"kubernetes.io/projected/2c7423c0-ed5e-4524-b955-b050a3bf286c-kube-api-access-rcx4j\") pod \"test-curl-7760-139936997725216\" (UID: \"2c7423c0-ed5e-4524-b955-b050a3bf286c\") " pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:33.654290 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.654266 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76b16151-2053-4941-833b-7c0748667372" path="/var/lib/kubelet/pods/76b16151-2053-4941-833b-7c0748667372/volumes" Jul 20 05:48:33.794067 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.794041 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:33.927857 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:33.927832 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7760-139936997725216"] Jul 20 05:48:33.930007 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:48:33.929976 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c7423c0_ed5e_4524_b955_b050a3bf286c.slice/crio-24bbbc923e4ae746abdac82b0a9afe37f21859d3b514cfb31a7cf3efa55844a5 WatchSource:0}: Error finding container 24bbbc923e4ae746abdac82b0a9afe37f21859d3b514cfb31a7cf3efa55844a5: Status 404 returned error can't find the container with id 24bbbc923e4ae746abdac82b0a9afe37f21859d3b514cfb31a7cf3efa55844a5 Jul 20 05:48:34.153880 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:34.153851 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:34.154089 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:34.153903 2576 scope.go:117] "RemoveContainer" containerID="ccfdb840c19b1a0474c84ca49a00d027e4f4b835acbe9640c996c6b5bd625f0b" Jul 20 05:48:34.155274 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:34.155147 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7760-139936997725216" event={"ID":"2c7423c0-ed5e-4524-b955-b050a3bf286c","Type":"ContainerStarted","Data":"24bbbc923e4ae746abdac82b0a9afe37f21859d3b514cfb31a7cf3efa55844a5"} Jul 20 05:48:35.163204 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:35.163167 2576 generic.go:356] "Generic (PLEG): container finished" podID="2c7423c0-ed5e-4524-b955-b050a3bf286c" containerID="6ef57961aa24ee8b212e7f70d2a755efe1eb4bac99118926f927c3e4a5d5bb5f" exitCode=0 Jul 20 05:48:35.163561 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:35.163247 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7760-139936997725216" event={"ID":"2c7423c0-ed5e-4524-b955-b050a3bf286c","Type":"ContainerDied","Data":"6ef57961aa24ee8b212e7f70d2a755efe1eb4bac99118926f927c3e4a5d5bb5f"} Jul 20 05:48:36.309465 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.309437 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:36.331160 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.331136 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7760-139936997725216_2c7423c0-ed5e-4524-b955-b050a3bf286c/test-curl-7760-139936997725216/0.log" Jul 20 05:48:36.337251 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.337226 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcx4j\" (UniqueName: \"kubernetes.io/projected/2c7423c0-ed5e-4524-b955-b050a3bf286c-kube-api-access-rcx4j\") pod \"2c7423c0-ed5e-4524-b955-b050a3bf286c\" (UID: \"2c7423c0-ed5e-4524-b955-b050a3bf286c\") " Jul 20 05:48:36.339078 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.339056 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c7423c0-ed5e-4524-b955-b050a3bf286c-kube-api-access-rcx4j" (OuterVolumeSpecName: "kube-api-access-rcx4j") pod "2c7423c0-ed5e-4524-b955-b050a3bf286c" (UID: "2c7423c0-ed5e-4524-b955-b050a3bf286c"). InnerVolumeSpecName "kube-api-access-rcx4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 05:48:36.356652 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.356613 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7760-139936997725216"] Jul 20 05:48:36.359634 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.359610 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7760-139936997725216"] Jul 20 05:48:36.438623 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.438571 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcx4j\" (UniqueName: \"kubernetes.io/projected/2c7423c0-ed5e-4524-b955-b050a3bf286c-kube-api-access-rcx4j\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:48:36.547801 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.547769 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm"] Jul 20 05:48:36.548225 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.548211 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c7423c0-ed5e-4524-b955-b050a3bf286c" containerName="test-curl-7760-139936997725216" Jul 20 05:48:36.548225 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.548227 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c7423c0-ed5e-4524-b955-b050a3bf286c" containerName="test-curl-7760-139936997725216" Jul 20 05:48:36.548331 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.548304 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c7423c0-ed5e-4524-b955-b050a3bf286c" containerName="test-curl-7760-139936997725216" Jul 20 05:48:36.551387 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.551371 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm" Jul 20 05:48:36.553933 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.553749 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:48:36.557634 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.557610 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm"] Jul 20 05:48:36.640744 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.640709 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf23d296-e191-461d-b668-5374664c03b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-851aef-df2gm\" (UID: \"cf23d296-e191-461d-b668-5374664c03b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm" Jul 20 05:48:36.741934 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.741907 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf23d296-e191-461d-b668-5374664c03b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-851aef-df2gm\" (UID: \"cf23d296-e191-461d-b668-5374664c03b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm" Jul 20 05:48:36.742519 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.742498 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf23d296-e191-461d-b668-5374664c03b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-851aef-df2gm\" (UID: \"cf23d296-e191-461d-b668-5374664c03b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm" Jul 20 05:48:36.872977 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:36.872920 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm" Jul 20 05:48:37.013530 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:37.013500 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm"] Jul 20 05:48:37.016550 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:48:37.016523 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf23d296_e191_461d_b668_5374664c03b2.slice/crio-fdc02951c1076c2cd82fc98f815e91657847c5a65f125b8543f845e59da438f0 WatchSource:0}: Error finding container fdc02951c1076c2cd82fc98f815e91657847c5a65f125b8543f845e59da438f0: Status 404 returned error can't find the container with id fdc02951c1076c2cd82fc98f815e91657847c5a65f125b8543f845e59da438f0 Jul 20 05:48:37.028529 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:37.028497 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:37.126033 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:37.126003 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:37.177294 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:37.177261 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm" event={"ID":"cf23d296-e191-461d-b668-5374664c03b2","Type":"ContainerStarted","Data":"fdc02951c1076c2cd82fc98f815e91657847c5a65f125b8543f845e59da438f0"} Jul 20 05:48:37.179111 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:37.179080 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:37.179236 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:37.179123 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24bbbc923e4ae746abdac82b0a9afe37f21859d3b514cfb31a7cf3efa55844a5" Jul 20 05:48:37.190002 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:48:37.189973 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7760-139936997725216\" is forbidden: User \"system:node:ip-10-0-128-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-33.ec2.internal' and this object" podUID="2c7423c0-ed5e-4524-b955-b050a3bf286c" pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:37.652415 ip-10-0-128-33 kubenswrapper[2576]: E0720 05:48:37.652328 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7760-139936997725216\" is forbidden: User \"system:node:ip-10-0-128-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-33.ec2.internal' and this object" podUID="2c7423c0-ed5e-4524-b955-b050a3bf286c" pod="opendatahub/test-curl-7760-139936997725216" Jul 20 05:48:37.654323 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:37.654293 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c7423c0-ed5e-4524-b955-b050a3bf286c" path="/var/lib/kubelet/pods/2c7423c0-ed5e-4524-b955-b050a3bf286c/volumes" Jul 20 05:48:38.187240 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:38.187204 2576 generic.go:356] "Generic (PLEG): container finished" podID="cf23d296-e191-461d-b668-5374664c03b2" containerID="0ec40f0f3982af27a5c231fd921b6dd32f104d58cd5e5a5aa2a8a015f70f2873" exitCode=0 Jul 20 05:48:38.187421 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:38.187250 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm" event={"ID":"cf23d296-e191-461d-b668-5374664c03b2","Type":"ContainerDied","Data":"0ec40f0f3982af27a5c231fd921b6dd32f104d58cd5e5a5aa2a8a015f70f2873"} Jul 20 05:48:39.345738 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:39.345717 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm" Jul 20 05:48:39.465480 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:39.465396 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf23d296-e191-461d-b668-5374664c03b2-maas-api-service-ca\") pod \"cf23d296-e191-461d-b668-5374664c03b2\" (UID: \"cf23d296-e191-461d-b668-5374664c03b2\") " Jul 20 05:48:39.465737 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:39.465714 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf23d296-e191-461d-b668-5374664c03b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cf23d296-e191-461d-b668-5374664c03b2" (UID: "cf23d296-e191-461d-b668-5374664c03b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:48:39.465814 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:39.465801 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf23d296-e191-461d-b668-5374664c03b2-maas-api-service-ca\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:48:40.200356 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:40.200330 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm" Jul 20 05:48:40.200531 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:40.200371 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm" event={"ID":"cf23d296-e191-461d-b668-5374664c03b2","Type":"ContainerDied","Data":"fdc02951c1076c2cd82fc98f815e91657847c5a65f125b8543f845e59da438f0"} Jul 20 05:48:40.200531 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:40.200417 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fdc02951c1076c2cd82fc98f815e91657847c5a65f125b8543f845e59da438f0" Jul 20 05:48:46.947947 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:46.947911 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:46.948567 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:46.948545 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m"] Jul 20 05:48:46.951192 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:46.951162 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-851aef-29742105-c8t6m"] Jul 20 05:48:47.653214 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:47.653185 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f3f840d-6351-44ec-90f8-4019d66628e0" path="/var/lib/kubelet/pods/5f3f840d-6351-44ec-90f8-4019d66628e0/volumes" Jul 20 05:48:51.622453 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:51.622421 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:51.635815 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:51.635768 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm"] Jul 20 05:48:51.638616 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:51.638595 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-851aef-df2gm"] Jul 20 05:48:51.654092 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:51.654070 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf23d296-e191-461d-b668-5374664c03b2" path="/var/lib/kubelet/pods/cf23d296-e191-461d-b668-5374664c03b2/volumes" Jul 20 05:48:52.025076 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:52.025047 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:58.040581 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.040544 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:58.149133 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.149101 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:58.315112 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.315030 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:58.568190 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.568110 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:58.680229 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.680190 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk"] Jul 20 05:48:58.680679 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.680660 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf23d296-e191-461d-b668-5374664c03b2" containerName="revoke-keys" Jul 20 05:48:58.680770 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.680683 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf23d296-e191-461d-b668-5374664c03b2" containerName="revoke-keys" Jul 20 05:48:58.680855 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.680840 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf23d296-e191-461d-b668-5374664c03b2" containerName="revoke-keys" Jul 20 05:48:58.684146 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.684124 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk" Jul 20 05:48:58.686733 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.686698 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-w55gj\"" Jul 20 05:48:58.689997 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.689972 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk"] Jul 20 05:48:58.752251 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.752213 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b32c651c-a8ad-4829-b6ad-7e02788aef70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk\" (UID: \"b32c651c-a8ad-4829-b6ad-7e02788aef70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk" Jul 20 05:48:58.853048 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.852925 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b32c651c-a8ad-4829-b6ad-7e02788aef70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk\" (UID: \"b32c651c-a8ad-4829-b6ad-7e02788aef70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk" Jul 20 05:48:58.853541 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.853522 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b32c651c-a8ad-4829-b6ad-7e02788aef70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk\" (UID: \"b32c651c-a8ad-4829-b6ad-7e02788aef70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk" Jul 20 05:48:58.996854 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:58.996824 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk" Jul 20 05:48:59.130374 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:59.130342 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk"] Jul 20 05:48:59.131735 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:48:59.131703 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32c651c_a8ad_4829_b6ad_7e02788aef70.slice/crio-fc6250ebfd99c5efb0c4fd37b7f67373cdb6d1c75870faff945261f5c343bcfd WatchSource:0}: Error finding container fc6250ebfd99c5efb0c4fd37b7f67373cdb6d1c75870faff945261f5c343bcfd: Status 404 returned error can't find the container with id fc6250ebfd99c5efb0c4fd37b7f67373cdb6d1c75870faff945261f5c343bcfd Jul 20 05:48:59.283085 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:59.283051 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:48:59.315232 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:59.315200 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk" event={"ID":"b32c651c-a8ad-4829-b6ad-7e02788aef70","Type":"ContainerStarted","Data":"fc6250ebfd99c5efb0c4fd37b7f67373cdb6d1c75870faff945261f5c343bcfd"} Jul 20 05:48:59.379924 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:48:59.379862 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:49:00.322644 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:00.322613 2576 generic.go:356] "Generic (PLEG): container finished" podID="b32c651c-a8ad-4829-b6ad-7e02788aef70" containerID="27dac1edd1485948dc9e405764603ad74c6bacf49abcc16d2e97ebb9121433ac" exitCode=0 Jul 20 05:49:00.323106 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:00.322707 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk" event={"ID":"b32c651c-a8ad-4829-b6ad-7e02788aef70","Type":"ContainerDied","Data":"27dac1edd1485948dc9e405764603ad74c6bacf49abcc16d2e97ebb9121433ac"} Jul 20 05:49:01.473857 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:01.473829 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk" Jul 20 05:49:01.581185 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:01.581093 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b32c651c-a8ad-4829-b6ad-7e02788aef70-maas-api-service-ca\") pod \"b32c651c-a8ad-4829-b6ad-7e02788aef70\" (UID: \"b32c651c-a8ad-4829-b6ad-7e02788aef70\") " Jul 20 05:49:01.581424 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:01.581402 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b32c651c-a8ad-4829-b6ad-7e02788aef70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b32c651c-a8ad-4829-b6ad-7e02788aef70" (UID: "b32c651c-a8ad-4829-b6ad-7e02788aef70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 05:49:01.682835 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:01.682806 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b32c651c-a8ad-4829-b6ad-7e02788aef70-maas-api-service-ca\") on node \"ip-10-0-128-33.ec2.internal\" DevicePath \"\"" Jul 20 05:49:02.335703 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:02.335672 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk" Jul 20 05:49:02.335872 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:02.335705 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk" event={"ID":"b32c651c-a8ad-4829-b6ad-7e02788aef70","Type":"ContainerDied","Data":"fc6250ebfd99c5efb0c4fd37b7f67373cdb6d1c75870faff945261f5c343bcfd"} Jul 20 05:49:02.335872 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:02.335754 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc6250ebfd99c5efb0c4fd37b7f67373cdb6d1c75870faff945261f5c343bcfd" Jul 20 05:49:09.091383 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:09.091346 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:49:09.092569 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:09.092542 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82cedd-29742105-f9x7p"] Jul 20 05:49:09.096160 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:09.096131 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82cedd-29742105-f9x7p"] Jul 20 05:49:09.658314 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:09.658277 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c19e84d-010e-46e5-be24-8ac740f921bc" path="/var/lib/kubelet/pods/8c19e84d-010e-46e5-be24-8ac740f921bc/volumes" Jul 20 05:49:13.749347 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:13.749314 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:49:13.752037 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:13.752009 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk"] Jul 20 05:49:13.754466 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:13.754444 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-82cedd-5dvhk"] Jul 20 05:49:14.170854 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:14.170777 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:49:15.654098 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:15.654062 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b32c651c-a8ad-4829-b6ad-7e02788aef70" path="/var/lib/kubelet/pods/b32c651c-a8ad-4829-b6ad-7e02788aef70/volumes" Jul 20 05:49:18.016028 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:18.016001 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:49:18.020093 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:18.020071 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:49:18.022856 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:18.022839 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:49:18.027070 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:18.027046 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85lxq_461b67c2-73dc-4b07-9b52-f102773c0aef/ovn-acl-logging/0.log" Jul 20 05:49:20.076732 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:20.076700 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:49:20.132590 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:20.132561 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:49:20.344062 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:20.343974 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:49:20.581343 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:20.581313 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:49:26.909901 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:26.909875 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b4f66bf88-dmxd7_36285f66-63b4-4188-befb-88219bed772f/manager/0.log" Jul 20 05:49:27.269994 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:27.269944 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54dff8fcf7-2c2gd_0f0bfbb7-fe06-4730-8a34-41368c5d39ce/copy-manifests/0.log" Jul 20 05:49:27.276940 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:27.276922 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54dff8fcf7-2c2gd_0f0bfbb7-fe06-4730-8a34-41368c5d39ce/manager/0.log" Jul 20 05:49:28.035791 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:28.035759 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742075-wxcvr_62cc1edf-1bf4-4310-ace5-86f79902a2cb/cleanup/0.log" Jul 20 05:49:28.153524 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:28.153491 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742090-dwrvh_d84a4ee6-0f61-4d57-b17e-3b47d8d2c477/cleanup/0.log" Jul 20 05:49:28.275663 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:28.275639 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742105-jclv4_899a75a2-63c1-4423-92c7-63b236c82dde/cleanup/0.log" Jul 20 05:49:28.397252 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:28.397175 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-jmkg6_d05b2693-721a-4b8b-99e7-c439a57ea19d/postgres/0.log" Jul 20 05:49:29.195662 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.195635 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph_e245d100-85e9-47d3-bda0-e7f8367b2ad7/util/0.log" Jul 20 05:49:29.202116 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.202088 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph_e245d100-85e9-47d3-bda0-e7f8367b2ad7/pull/0.log" Jul 20 05:49:29.208438 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.208412 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph_e245d100-85e9-47d3-bda0-e7f8367b2ad7/extract/0.log" Jul 20 05:49:29.328682 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.328648 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78_e040e05c-b612-48e9-a442-53b1505c7187/extract/0.log" Jul 20 05:49:29.335549 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.335522 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78_e040e05c-b612-48e9-a442-53b1505c7187/util/0.log" Jul 20 05:49:29.342125 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.342109 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78_e040e05c-b612-48e9-a442-53b1505c7187/pull/0.log" Jul 20 05:49:29.459121 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.459054 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn_47df011b-30b9-475e-8bdb-f28c2a1cfe8e/extract/0.log" Jul 20 05:49:29.465195 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.465169 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn_47df011b-30b9-475e-8bdb-f28c2a1cfe8e/util/0.log" Jul 20 05:49:29.471540 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.471518 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn_47df011b-30b9-475e-8bdb-f28c2a1cfe8e/pull/0.log" Jul 20 05:49:29.589827 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.589801 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74_3a4ced7e-c8df-47fb-86c6-b65d31593b06/util/0.log" Jul 20 05:49:29.596406 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.596382 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74_3a4ced7e-c8df-47fb-86c6-b65d31593b06/pull/0.log" Jul 20 05:49:29.603071 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.603055 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74_3a4ced7e-c8df-47fb-86c6-b65d31593b06/extract/0.log" Jul 20 05:49:29.853249 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.853224 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-plxhr_5f06de2c-28ab-4315-8dc1-9aaa4e1a0f6f/manager/0.log" Jul 20 05:49:29.978687 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:29.978650 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-k6jlw_66e63abf-0805-462f-8808-117879a8ad7b/manager/0.log" Jul 20 05:49:30.093117 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:30.093089 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-q2782_9646f70b-dda3-496b-98a6-6e8e33e617d8/kuadrant-console-plugin/0.log" Jul 20 05:49:30.217800 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:30.217729 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zkzpl_a4d8fa3a-2526-4081-a7b8-7154a9efb8fa/registry-server/0.log" Jul 20 05:49:30.988308 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:30.988277 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4w7zs9_eae43db9-d7d1-4eb1-abd0-0da24b91df53/istio-proxy/0.log" Jul 20 05:49:31.464399 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:31.464321 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-r4zbh_88b632e5-2858-48b7-817a-5704fe6f3469/istio-proxy/1.log" Jul 20 05:49:31.590303 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:31.590275 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-p7lst_4c0b6951-8797-43f2-b26e-0d9b5ce04e71/payload-pre-processing/0.log" Jul 20 05:49:31.713496 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:31.713467 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-fmsw5_0aa67061-8af2-48f0-9c2c-ca47229be185/payload-processing/0.log" Jul 20 05:49:31.837587 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:31.837558 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64bbb4d846-ftbct_1877c3b7-fa1e-49c5-a296-ef9478df42e8/router/0.log" Jul 20 05:49:32.188929 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:32.188852 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-7rp2m_4e3b621d-6ba5-4d6b-a91d-6d8f9c01b199/main/0.log" Jul 20 05:49:32.548573 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:32.548543 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5hx4h_92e8e2cd-3a6e-4fc8-b981-d8c0a11a32d5/main/0.log" Jul 20 05:49:38.348949 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.348911 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g8fln/must-gather-rlr5w"] Jul 20 05:49:38.349395 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.349381 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b32c651c-a8ad-4829-b6ad-7e02788aef70" containerName="revoke-keys" Jul 20 05:49:38.349439 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.349397 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b32c651c-a8ad-4829-b6ad-7e02788aef70" containerName="revoke-keys" Jul 20 05:49:38.349474 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.349467 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b32c651c-a8ad-4829-b6ad-7e02788aef70" containerName="revoke-keys" Jul 20 05:49:38.353086 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.353070 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8fln/must-gather-rlr5w" Jul 20 05:49:38.355785 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.355762 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g8fln\"/\"default-dockercfg-c52p4\"" Jul 20 05:49:38.355922 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.355804 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g8fln\"/\"openshift-service-ca.crt\"" Jul 20 05:49:38.355922 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.355809 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g8fln\"/\"kube-root-ca.crt\"" Jul 20 05:49:38.362756 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.362733 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8fln/must-gather-rlr5w"] Jul 20 05:49:38.363366 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.363340 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:49:38.429272 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.429244 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpbbc\" (UniqueName: \"kubernetes.io/projected/c5e55103-fe29-4eb2-a68c-ec85857aa026-kube-api-access-zpbbc\") pod \"must-gather-rlr5w\" (UID: \"c5e55103-fe29-4eb2-a68c-ec85857aa026\") " pod="openshift-must-gather-g8fln/must-gather-rlr5w" Jul 20 05:49:38.429455 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.429280 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c5e55103-fe29-4eb2-a68c-ec85857aa026-must-gather-output\") pod \"must-gather-rlr5w\" (UID: \"c5e55103-fe29-4eb2-a68c-ec85857aa026\") " pod="openshift-must-gather-g8fln/must-gather-rlr5w" Jul 20 05:49:38.463256 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.463223 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:49:38.530143 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.530114 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpbbc\" (UniqueName: \"kubernetes.io/projected/c5e55103-fe29-4eb2-a68c-ec85857aa026-kube-api-access-zpbbc\") pod \"must-gather-rlr5w\" (UID: \"c5e55103-fe29-4eb2-a68c-ec85857aa026\") " pod="openshift-must-gather-g8fln/must-gather-rlr5w" Jul 20 05:49:38.530314 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.530150 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c5e55103-fe29-4eb2-a68c-ec85857aa026-must-gather-output\") pod \"must-gather-rlr5w\" (UID: \"c5e55103-fe29-4eb2-a68c-ec85857aa026\") " pod="openshift-must-gather-g8fln/must-gather-rlr5w" Jul 20 05:49:38.530547 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.530526 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c5e55103-fe29-4eb2-a68c-ec85857aa026-must-gather-output\") pod \"must-gather-rlr5w\" (UID: \"c5e55103-fe29-4eb2-a68c-ec85857aa026\") " pod="openshift-must-gather-g8fln/must-gather-rlr5w" Jul 20 05:49:38.538702 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.538680 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpbbc\" (UniqueName: \"kubernetes.io/projected/c5e55103-fe29-4eb2-a68c-ec85857aa026-kube-api-access-zpbbc\") pod \"must-gather-rlr5w\" (UID: \"c5e55103-fe29-4eb2-a68c-ec85857aa026\") " pod="openshift-must-gather-g8fln/must-gather-rlr5w" Jul 20 05:49:38.664192 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.664107 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8fln/must-gather-rlr5w" Jul 20 05:49:38.807319 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:38.807294 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8fln/must-gather-rlr5w"] Jul 20 05:49:38.809559 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:49:38.809530 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5e55103_fe29_4eb2_a68c_ec85857aa026.slice/crio-5ccf6944347d98dae1f88a9ddf80bdd3f764624bf05e3bde37d0849ce3f4f534 WatchSource:0}: Error finding container 5ccf6944347d98dae1f88a9ddf80bdd3f764624bf05e3bde37d0849ce3f4f534: Status 404 returned error can't find the container with id 5ccf6944347d98dae1f88a9ddf80bdd3f764624bf05e3bde37d0849ce3f4f534 Jul 20 05:49:39.565152 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:39.565111 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8fln/must-gather-rlr5w" event={"ID":"c5e55103-fe29-4eb2-a68c-ec85857aa026","Type":"ContainerStarted","Data":"5ccf6944347d98dae1f88a9ddf80bdd3f764624bf05e3bde37d0849ce3f4f534"} Jul 20 05:49:40.576865 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:40.576828 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8fln/must-gather-rlr5w" event={"ID":"c5e55103-fe29-4eb2-a68c-ec85857aa026","Type":"ContainerStarted","Data":"f32e545e7c6f2469b6d56aa96179bdf6225c58b1babbbb0fb4e1d9b9add6d321"} Jul 20 05:49:40.577433 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:40.577407 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8fln/must-gather-rlr5w" event={"ID":"c5e55103-fe29-4eb2-a68c-ec85857aa026","Type":"ContainerStarted","Data":"851e3d36c7b27a73558c68a1ce884e5403dc3ace00ce3d781a528befec9de74d"} Jul 20 05:49:40.594250 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:40.594202 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g8fln/must-gather-rlr5w" podStartSLOduration=1.793002162 podStartE2EDuration="2.594187599s" totalImagesPullingTime="801.185437ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 05:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:49:40.592835831 +0000 UTC m=+3023.639362738" watchObservedRunningTime="2026-07-20 05:49:40.594187599 +0000 UTC m=+3023.640714504" Jul 20 05:49:41.190845 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:41.190813 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dh2cl_a9dd925c-ebe0-4584-a59a-bcc23cb8a69b/global-pull-secret-syncer/0.log" Jul 20 05:49:41.277897 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:41.277865 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" Jul 20 05:49:41.354240 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:41.354213 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-33.ec2.internal_a8c6bf12d5d282ff52e7d38919e9afae/haproxy/0.log" Jul 20 05:49:45.961760 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:45.961731 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph_e245d100-85e9-47d3-bda0-e7f8367b2ad7/extract/0.log" Jul 20 05:49:45.985449 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:45.985379 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph_e245d100-85e9-47d3-bda0-e7f8367b2ad7/util/0.log" Jul 20 05:49:46.006869 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.006845 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ltnph_e245d100-85e9-47d3-bda0-e7f8367b2ad7/pull/0.log" Jul 20 05:49:46.038158 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.038129 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78_e040e05c-b612-48e9-a442-53b1505c7187/extract/0.log" Jul 20 05:49:46.058590 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.058564 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78_e040e05c-b612-48e9-a442-53b1505c7187/util/0.log" Jul 20 05:49:46.081575 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.081549 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pd78_e040e05c-b612-48e9-a442-53b1505c7187/pull/0.log" Jul 20 05:49:46.108625 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.108593 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn_47df011b-30b9-475e-8bdb-f28c2a1cfe8e/extract/0.log" Jul 20 05:49:46.143040 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.143011 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn_47df011b-30b9-475e-8bdb-f28c2a1cfe8e/util/0.log" Jul 20 05:49:46.163628 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.163586 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dpwrn_47df011b-30b9-475e-8bdb-f28c2a1cfe8e/pull/0.log" Jul 20 05:49:46.193187 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.193153 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74_3a4ced7e-c8df-47fb-86c6-b65d31593b06/extract/0.log" Jul 20 05:49:46.212896 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.212774 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74_3a4ced7e-c8df-47fb-86c6-b65d31593b06/util/0.log" Jul 20 05:49:46.234417 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.234322 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdq74_3a4ced7e-c8df-47fb-86c6-b65d31593b06/pull/0.log" Jul 20 05:49:46.611659 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.611620 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-plxhr_5f06de2c-28ab-4315-8dc1-9aaa4e1a0f6f/manager/0.log" Jul 20 05:49:46.651395 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.651367 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-k6jlw_66e63abf-0805-462f-8808-117879a8ad7b/manager/0.log" Jul 20 05:49:46.681470 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.681441 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-q2782_9646f70b-dda3-496b-98a6-6e8e33e617d8/kuadrant-console-plugin/0.log" Jul 20 05:49:46.728409 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:46.728379 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zkzpl_a4d8fa3a-2526-4081-a7b8-7154a9efb8fa/registry-server/0.log" Jul 20 05:49:48.827386 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:48.827355 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-td56c_4d64666c-dff4-424e-a774-9f90c94a206c/cluster-monitoring-operator/0.log" Jul 20 05:49:48.853912 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:48.853879 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-qvbb9_5f5274c4-78fe-4204-95e9-dc69ae66284f/kube-state-metrics/0.log" Jul 20 05:49:48.871534 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:48.871506 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-qvbb9_5f5274c4-78fe-4204-95e9-dc69ae66284f/kube-rbac-proxy-main/0.log" Jul 20 05:49:48.892452 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:48.892418 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-qvbb9_5f5274c4-78fe-4204-95e9-dc69ae66284f/kube-rbac-proxy-self/0.log" Jul 20 05:49:48.916543 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:48.916511 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6dd68dc75-smzfb_c7285c72-07f0-41f6-aa19-d40eeb35f04c/metrics-server/0.log" Jul 20 05:49:49.133161 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:49.133071 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mh57v_7eab4a6c-df4e-4a91-8214-64489829bf9f/node-exporter/0.log" Jul 20 05:49:49.154330 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:49.154297 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mh57v_7eab4a6c-df4e-4a91-8214-64489829bf9f/kube-rbac-proxy/0.log" Jul 20 05:49:49.175903 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:49.175878 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mh57v_7eab4a6c-df4e-4a91-8214-64489829bf9f/init-textfile/0.log" Jul 20 05:49:49.203475 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:49.203443 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-7jpfh_5f1cf2be-098d-4985-bde4-081f4388c57b/kube-rbac-proxy-main/0.log" Jul 20 05:49:49.232318 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:49.232289 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-7jpfh_5f1cf2be-098d-4985-bde4-081f4388c57b/kube-rbac-proxy-self/0.log" Jul 20 05:49:49.254975 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:49.254925 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-7jpfh_5f1cf2be-098d-4985-bde4-081f4388c57b/openshift-state-metrics/0.log" Jul 20 05:49:49.594594 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:49.594562 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ffb49c748-9lkrz_9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0/thanos-query/0.log" Jul 20 05:49:49.615801 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:49.615775 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ffb49c748-9lkrz_9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0/kube-rbac-proxy-web/0.log" Jul 20 05:49:49.635969 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:49.635921 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ffb49c748-9lkrz_9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0/kube-rbac-proxy/0.log" Jul 20 05:49:49.656169 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:49.656128 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ffb49c748-9lkrz_9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0/prom-label-proxy/0.log" Jul 20 05:49:49.675591 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:49.675566 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ffb49c748-9lkrz_9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0/kube-rbac-proxy-rules/0.log" Jul 20 05:49:49.695004 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:49.694972 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ffb49c748-9lkrz_9f4b4bf0-c6bf-4027-b9e5-e81ea9f016e0/kube-rbac-proxy-metrics/0.log" Jul 20 05:49:51.285373 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.285322 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9"] Jul 20 05:49:51.299245 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.299219 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.299804 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.299772 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9"] Jul 20 05:49:51.390997 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.389656 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-sys\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.390997 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.389749 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-proc\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.390997 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.389882 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-podres\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.390997 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.389977 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6mbk\" (UniqueName: \"kubernetes.io/projected/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-kube-api-access-v6mbk\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.390997 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.390033 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-lib-modules\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.455226 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.455196 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/1.log" Jul 20 05:49:51.462166 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.462130 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-q8k7n_240fe5fe-623e-4495-a594-63f72c28f6bd/console-operator/2.log" Jul 20 05:49:51.491266 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.491238 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-lib-modules\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.491415 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.491313 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-sys\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.491415 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.491373 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-proc\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.491496 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.491422 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-sys\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.491496 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.491430 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-lib-modules\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.491496 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.491435 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-podres\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.491640 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.491497 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-proc\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.491640 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.491508 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-podres\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.491640 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.491561 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6mbk\" (UniqueName: \"kubernetes.io/projected/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-kube-api-access-v6mbk\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.501750 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.501727 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6mbk\" (UniqueName: \"kubernetes.io/projected/3a8c90bd-ef27-4ae1-a205-8657bca57cfa-kube-api-access-v6mbk\") pod \"perf-node-gather-daemonset-r4zn9\" (UID: \"3a8c90bd-ef27-4ae1-a205-8657bca57cfa\") " pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:51.613156 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:51.613080 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:52.000392 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:52.000364 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9"] Jul 20 05:49:52.002596 ip-10-0-128-33 kubenswrapper[2576]: W0720 05:49:52.002564 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3a8c90bd_ef27_4ae1_a205_8657bca57cfa.slice/crio-49a396fb1bf0d03c814f9336ea3be42e69991f0cef546df1191adcb0662cecb4 WatchSource:0}: Error finding container 49a396fb1bf0d03c814f9336ea3be42e69991f0cef546df1191adcb0662cecb4: Status 404 returned error can't find the container with id 49a396fb1bf0d03c814f9336ea3be42e69991f0cef546df1191adcb0662cecb4 Jul 20 05:49:52.008301 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:52.008269 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-577bc74b9b-ht5q5_35c45dc4-778b-4721-8ecf-696db95c8dab/console/0.log" Jul 20 05:49:52.043535 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:52.043467 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-z65tl_91155ba9-0eab-446c-a27e-a2c2d7321584/download-server/0.log" Jul 20 05:49:52.628837 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:52.628806 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-9mdfj_8bf73698-17d9-4e7f-b633-d7d07e7df680/volume-data-source-validator/0.log" Jul 20 05:49:52.696936 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:52.696893 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" event={"ID":"3a8c90bd-ef27-4ae1-a205-8657bca57cfa","Type":"ContainerStarted","Data":"e28bb937fe859f97d4d22ea75254a48e2bb4decdde5343f3b1e879ba648978f7"} Jul 20 05:49:52.697205 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:52.697178 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:52.697325 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:52.697229 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" event={"ID":"3a8c90bd-ef27-4ae1-a205-8657bca57cfa","Type":"ContainerStarted","Data":"49a396fb1bf0d03c814f9336ea3be42e69991f0cef546df1191adcb0662cecb4"} Jul 20 05:49:52.715344 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:52.715282 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" podStartSLOduration=1.715263207 podStartE2EDuration="1.715263207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 05:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 05:49:52.711702715 +0000 UTC m=+3035.758229642" watchObservedRunningTime="2026-07-20 05:49:52.715263207 +0000 UTC m=+3035.761790114" Jul 20 05:49:53.626896 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:53.626863 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cvb6r_6a13f44c-570d-4fa6-96c6-bcdead54f7ec/dns/0.log" Jul 20 05:49:53.648770 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:53.648734 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cvb6r_6a13f44c-570d-4fa6-96c6-bcdead54f7ec/kube-rbac-proxy/0.log" Jul 20 05:49:53.742266 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:53.742241 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-js7gh_1edae688-9ce1-49e7-9a37-5d782b643b36/dns-node-resolver/0.log" Jul 20 05:49:54.277973 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:54.277937 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7899876bcf-zm2cl_c73e5531-c6cc-4512-bdef-f859d992f7ec/registry/0.log" Jul 20 05:49:54.318111 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:54.318084 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p29tq_42007563-e764-4689-a28f-872b48cd1e8b/node-ca/0.log" Jul 20 05:49:55.231340 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:55.231304 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4w7zs9_eae43db9-d7d1-4eb1-abd0-0da24b91df53/istio-proxy/0.log" Jul 20 05:49:55.418811 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:55.418773 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-r4zbh_88b632e5-2858-48b7-817a-5704fe6f3469/istio-proxy/0.log" Jul 20 05:49:55.455636 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:55.455609 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-r4zbh_88b632e5-2858-48b7-817a-5704fe6f3469/istio-proxy/1.log" Jul 20 05:49:55.489981 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:55.489859 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-p7lst_4c0b6951-8797-43f2-b26e-0d9b5ce04e71/payload-pre-processing/0.log" Jul 20 05:49:55.520158 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:55.520125 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-fmsw5_0aa67061-8af2-48f0-9c2c-ca47229be185/payload-processing/0.log" Jul 20 05:49:55.540313 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:55.540285 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64bbb4d846-ftbct_1877c3b7-fa1e-49c5-a296-ef9478df42e8/router/0.log" Jul 20 05:49:56.243381 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:56.243350 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sf226_03c3ada9-2806-4b6e-9e90-ee27aed55433/serve-healthcheck-canary/0.log" Jul 20 05:49:56.702217 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:56.702117 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-c9l2r_9661ba55-2be6-499e-a1af-a5c0cb54f376/insights-operator/0.log" Jul 20 05:49:56.703170 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:56.703149 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-c9l2r_9661ba55-2be6-499e-a1af-a5c0cb54f376/insights-operator/1.log" Jul 20 05:49:56.855310 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:56.855278 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vddgs_a629203c-afcc-4207-8313-26f75cdc20ad/kube-rbac-proxy/0.log" Jul 20 05:49:56.874786 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:56.874759 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vddgs_a629203c-afcc-4207-8313-26f75cdc20ad/exporter/0.log" Jul 20 05:49:56.895401 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:56.895369 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vddgs_a629203c-afcc-4207-8313-26f75cdc20ad/extractor/0.log" Jul 20 05:49:58.718457 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:58.718425 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g8fln/perf-node-gather-daemonset-r4zn9" Jul 20 05:49:59.289352 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:59.289322 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b4f66bf88-dmxd7_36285f66-63b4-4188-befb-88219bed772f/manager/0.log" Jul 20 05:49:59.406653 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:59.406625 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54dff8fcf7-2c2gd_0f0bfbb7-fe06-4730-8a34-41368c5d39ce/manager/0.log" Jul 20 05:49:59.426173 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:49:59.426148 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54dff8fcf7-2c2gd_0f0bfbb7-fe06-4730-8a34-41368c5d39ce/copy-manifests/0.log" Jul 20 05:50:00.946655 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:00.946618 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-676c6f859f-cv6hb_989baa4d-2079-4239-a916-7a3187dbf2cd/manager/0.log" Jul 20 05:50:01.823018 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:01.822983 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4w7zs9_eae43db9-d7d1-4eb1-abd0-0da24b91df53/istio-proxy/0.log" Jul 20 05:50:01.982832 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:01.982799 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-r4zbh_88b632e5-2858-48b7-817a-5704fe6f3469/istio-proxy/0.log" Jul 20 05:50:02.019861 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:02.019828 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-r4zbh_88b632e5-2858-48b7-817a-5704fe6f3469/istio-proxy/1.log" Jul 20 05:50:02.044421 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:02.044384 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-p7lst_4c0b6951-8797-43f2-b26e-0d9b5ce04e71/payload-pre-processing/0.log" Jul 20 05:50:02.069373 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:02.069340 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-fmsw5_0aa67061-8af2-48f0-9c2c-ca47229be185/payload-processing/0.log" Jul 20 05:50:02.081739 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:02.081659 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64bbb4d846-ftbct_1877c3b7-fa1e-49c5-a296-ef9478df42e8/router/0.log" Jul 20 05:50:03.285638 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:03.285612 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4w7zs9_eae43db9-d7d1-4eb1-abd0-0da24b91df53/istio-proxy/0.log" Jul 20 05:50:03.468107 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:03.468025 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-r4zbh_88b632e5-2858-48b7-817a-5704fe6f3469/istio-proxy/0.log" Jul 20 05:50:03.505262 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:03.505232 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-r4zbh_88b632e5-2858-48b7-817a-5704fe6f3469/istio-proxy/1.log" Jul 20 05:50:03.532157 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:03.532128 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-p7lst_4c0b6951-8797-43f2-b26e-0d9b5ce04e71/payload-pre-processing/0.log" Jul 20 05:50:03.559710 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:03.559680 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-fmsw5_0aa67061-8af2-48f0-9c2c-ca47229be185/payload-processing/0.log" Jul 20 05:50:03.573663 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:03.573636 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64bbb4d846-ftbct_1877c3b7-fa1e-49c5-a296-ef9478df42e8/router/0.log" Jul 20 05:50:06.727516 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:06.727478 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:50:06.775426 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:06.775387 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ljhn_f0811f4e-d3ae-47c3-a490-a6c9c2db3010/konnectivity-agent/0.log" limit=1024 Jul 20 05:50:07.551770 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:07.551742 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhssz_f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1/kube-multus-additional-cni-plugins/0.log" Jul 20 05:50:07.570942 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:07.570915 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhssz_f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1/egress-router-binary-copy/0.log" Jul 20 05:50:07.589810 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:07.589790 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhssz_f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1/cni-plugins/0.log" Jul 20 05:50:07.609453 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:07.609436 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhssz_f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1/bond-cni-plugin/0.log" Jul 20 05:50:07.627920 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:07.627901 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhssz_f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1/routeoverride-cni/0.log" Jul 20 05:50:07.650669 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:07.650642 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhssz_f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1/whereabouts-cni-bincopy/0.log" Jul 20 05:50:07.669471 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:07.669451 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lhssz_f954ae54-6ba3-4d8c-a6e1-180fa75fc3a1/whereabouts-cni/0.log" Jul 20 05:50:07.853628 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:07.853547 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lpkbr_dcad9ac3-69d3-4c84-9f0f-dc1f45e86bfe/kube-multus/0.log" Jul 20 05:50:07.924680 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:07.924653 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gnmjm_06a59c1d-4c72-423f-9e8d-27db0a919dba/network-metrics-daemon/0.log" Jul 20 05:50:07.941525 ip-10-0-128-33 kubenswrapper[2576]: I0720 05:50:07.941501 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gnmjm_06a59c1d-4c72-423f-9e8d-27db0a919dba/kube-rbac-proxy/0.log"