Jul 30 02:23:54.693848 ip-10-0-131-5 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 30 02:23:54.693859 ip-10-0-131-5 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 30 02:23:54.693887 ip-10-0-131-5 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 30 02:23:54.694183 ip-10-0-131-5 systemd[1]: Failed to start Kubernetes Kubelet. Jul 30 02:24:04.181761 ip-10-0-131-5 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 30 02:24:04.181781 ip-10-0-131-5 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e89cc7eb76dc45038b857598d7f3a9f1 -- Jul 30 02:25:52.202176 ip-10-0-131-5 systemd[1]: Starting Kubernetes Kubelet... Jul 30 02:25:52.638348 ip-10-0-131-5 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 02:25:52.638348 ip-10-0-131-5 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 30 02:25:52.638348 ip-10-0-131-5 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 02:25:52.638348 ip-10-0-131-5 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 30 02:25:52.638348 ip-10-0-131-5 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 02:25:52.640530 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.640421 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 30 02:25:52.644314 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644299 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 02:25:52.644314 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644314 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644318 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644321 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644324 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644327 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644330 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644336 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644340 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644343 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644345 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644348 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644351 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644354 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644356 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644359 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644362 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644364 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644372 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644375 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644378 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 02:25:52.644375 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644380 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644383 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644386 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644389 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644392 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644394 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644397 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644400 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644402 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644405 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644407 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644410 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644412 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644415 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644417 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644420 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644422 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644425 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644428 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644430 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 02:25:52.644858 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644433 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644440 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644443 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644445 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644448 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644450 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644452 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644455 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644457 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644460 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644462 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644465 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644468 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644470 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644473 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644476 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644479 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644482 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644484 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644487 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 02:25:52.645325 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644489 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644492 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644494 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644499 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644502 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644506 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644509 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644511 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644514 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644516 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644519 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644522 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644526 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644529 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644531 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644534 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644536 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644539 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644541 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 02:25:52.645824 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644544 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644546 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644549 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644551 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644554 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644557 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644560 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644563 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644566 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644568 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644571 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644573 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644576 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644578 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644581 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644583 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644585 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644588 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 02:25:52.646288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.644590 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 02:25:52.647245 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647232 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 02:25:52.647245 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647245 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647249 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647252 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647255 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647259 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647262 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647264 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647267 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647272 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647275 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647278 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647281 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647283 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647286 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647289 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647291 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647294 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647296 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647299 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647302 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 02:25:52.647300 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647305 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647308 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647311 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647313 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647316 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647319 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647321 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647324 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647326 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647329 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647331 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647334 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647336 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647339 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647341 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647344 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647347 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647350 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647352 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647354 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 02:25:52.647813 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647357 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647360 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647362 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647366 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647369 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647372 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647375 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647378 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647387 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647390 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647392 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647395 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647398 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647401 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647404 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647407 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647409 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647412 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647415 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 02:25:52.648298 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647419 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647422 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647425 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647428 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647431 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647434 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647436 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647439 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647441 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647444 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647446 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647449 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647452 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647454 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647457 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647460 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647462 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647465 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647468 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647471 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 02:25:52.648790 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647474 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647476 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647479 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647481 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647483 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647486 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647488 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647492 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647494 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647497 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647499 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647502 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647505 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647507 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647510 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647512 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647514 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647517 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.647519 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648513 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 02:25:52.649286 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648520 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648523 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648526 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648530 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648533 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648536 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648538 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648542 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648546 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648549 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648551 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648555 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648558 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648560 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648563 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648565 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648568 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648571 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648573 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 02:25:52.649781 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648576 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648579 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648582 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648584 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648587 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648589 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648592 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648596 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648599 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648602 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648605 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648608 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648612 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648615 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648618 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648621 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648623 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648627 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648630 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 02:25:52.650280 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648633 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648635 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648638 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648641 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648643 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648646 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648649 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648651 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648654 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648656 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648671 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648674 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648677 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648680 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648682 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648685 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648688 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648691 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648693 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648696 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 02:25:52.650747 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648699 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648701 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648704 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648706 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648709 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648711 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648714 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648716 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648719 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648721 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648724 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648727 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648729 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648732 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648734 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648737 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648739 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648742 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648745 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648747 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 02:25:52.651246 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648750 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648752 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648755 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648758 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648760 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648763 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648767 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648769 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648772 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648775 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648778 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648780 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648783 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648785 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648788 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648791 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648793 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648796 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648798 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.648801 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648869 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 30 02:25:52.651779 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648876 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648883 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648897 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648901 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648905 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648909 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648913 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648916 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648919 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648923 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648926 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648929 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648933 2566 flags.go:64] FLAG: --cgroup-root="" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648935 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648938 2566 flags.go:64] FLAG: --client-ca-file="" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648941 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648945 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648948 2566 flags.go:64] FLAG: --cluster-domain="" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648951 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648954 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648957 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648960 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648964 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 30 02:25:52.652336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648966 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648969 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648973 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648976 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648978 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648981 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648984 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648987 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648992 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648994 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.648997 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649000 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649002 2566 flags.go:64] FLAG: --enable-server="true" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649006 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649010 2566 flags.go:64] FLAG: --event-burst="100" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649013 2566 flags.go:64] FLAG: --event-qps="50" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649016 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649019 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649022 2566 flags.go:64] FLAG: --eviction-hard="" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649026 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649029 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649032 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649035 2566 flags.go:64] FLAG: --eviction-soft="" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649037 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649040 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 30 02:25:52.652901 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649043 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649046 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649049 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649052 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649054 2566 flags.go:64] FLAG: --feature-gates="" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649058 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649061 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649064 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649067 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649070 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649073 2566 flags.go:64] FLAG: --help="false" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649075 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-131-5.ec2.internal" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649078 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649081 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649084 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649087 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649090 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649093 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649096 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649098 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649101 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649104 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649110 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649113 2566 flags.go:64] FLAG: --kube-reserved="" Jul 30 02:25:52.653546 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649116 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649119 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649122 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649125 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649127 2566 flags.go:64] FLAG: --lock-file="" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649130 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649133 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649136 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649142 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649144 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649147 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649150 2566 flags.go:64] FLAG: --logging-format="text" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649153 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649156 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649159 2566 flags.go:64] FLAG: --manifest-url="" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649162 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649166 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649169 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649173 2566 flags.go:64] FLAG: --max-pods="110" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649176 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649179 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649181 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649185 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649187 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649190 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 30 02:25:52.654153 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649193 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649200 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649203 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649205 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649209 2566 flags.go:64] FLAG: --pod-cidr="" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649213 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649218 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649221 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649224 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649227 2566 flags.go:64] FLAG: --port="10250" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649230 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649233 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08bfe67c70313fa68" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649236 2566 flags.go:64] FLAG: --qos-reserved="" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649238 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649242 2566 flags.go:64] FLAG: --register-node="true" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649244 2566 flags.go:64] FLAG: --register-with-taints="" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649248 2566 flags.go:64] FLAG: --registry-burst="10" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649251 2566 flags.go:64] FLAG: --registry-qps="5" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649254 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649257 2566 flags.go:64] FLAG: --reserved-memory="" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649260 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649263 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649266 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649269 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649271 2566 flags.go:64] FLAG: --runonce="false" Jul 30 02:25:52.654805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649274 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649277 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649280 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649283 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649285 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649289 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649292 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649295 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649298 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649301 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649303 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649306 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649310 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649313 2566 flags.go:64] FLAG: --system-cgroups="" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649316 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649321 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649324 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649327 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649332 2566 flags.go:64] FLAG: --tls-min-version="" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649335 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649337 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649340 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649343 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649346 2566 flags.go:64] FLAG: --v="2" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649350 2566 flags.go:64] FLAG: --version="false" Jul 30 02:25:52.655458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649354 2566 flags.go:64] FLAG: --vmodule="" Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649357 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649360 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649469 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649473 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649476 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649479 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649481 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649485 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649490 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649493 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649496 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649499 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649501 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649504 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649506 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649509 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649512 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649514 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 02:25:52.656148 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649519 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649521 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649524 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649527 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649530 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649533 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649537 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649539 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649542 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649544 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649547 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649549 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649552 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649555 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649557 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649560 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649562 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649565 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649567 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649570 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 02:25:52.656607 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649572 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649575 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649577 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649580 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649582 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649585 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649588 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649590 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649593 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649595 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649598 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649602 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649606 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649609 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649612 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649616 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649619 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649621 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649625 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649628 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 02:25:52.657092 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649631 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649633 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649636 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649639 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649642 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649644 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649647 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649649 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649652 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649655 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649657 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649671 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649674 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649676 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649679 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649683 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649687 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649689 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649692 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 02:25:52.657577 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649695 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649697 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649700 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649703 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649705 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649710 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649712 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649715 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649717 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649720 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649723 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649728 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649730 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649733 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649736 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649738 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649741 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649743 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649746 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649748 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 02:25:52.658056 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649751 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649753 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649756 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.649758 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.649764 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.656855 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.656869 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.656888 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.656894 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.656961 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.656965 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.656969 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.656972 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.656975 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.656978 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 02:25:52.658550 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.656980 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.656983 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.656986 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.656988 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.656991 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.656994 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657003 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657006 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657009 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657012 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657015 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657017 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657020 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657023 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657025 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657028 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657030 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657033 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657035 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657038 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 02:25:52.658942 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657041 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657049 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657052 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657056 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657058 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657061 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657063 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657066 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657069 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657071 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657074 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657076 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657078 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657081 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657083 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657086 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657088 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657091 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657093 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657102 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 02:25:52.659476 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657104 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657107 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657110 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657112 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657115 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657118 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657120 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657123 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657125 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657128 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657130 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657133 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657136 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657140 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657144 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657147 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657150 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657152 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657155 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657157 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 02:25:52.660414 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657160 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657162 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657165 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657167 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657170 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657173 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657175 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657177 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657180 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657183 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657187 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657191 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657199 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657202 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657206 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657210 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657212 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657215 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657218 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 02:25:52.660955 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657220 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657224 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657226 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657229 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657231 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657234 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657236 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657239 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657242 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657245 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657247 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657250 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657252 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657255 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 02:25:52.661411 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.657259 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657409 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657414 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657417 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657420 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657422 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657425 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657427 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657430 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657433 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657435 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657438 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657446 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657448 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657451 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657453 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657456 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657458 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657461 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657464 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 02:25:52.661806 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657466 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657469 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657471 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657475 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657478 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657480 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657483 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657485 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657487 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657490 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657492 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657495 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657497 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657500 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657502 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657505 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657507 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657509 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657512 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 02:25:52.662288 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657514 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657517 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657520 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657522 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657524 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657527 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657535 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657538 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657540 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657543 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657547 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657550 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657553 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657556 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657559 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657562 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657564 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657567 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657570 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 02:25:52.662761 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657573 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657576 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657580 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657583 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657586 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657589 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657592 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657594 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657597 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657599 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657602 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657604 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657607 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657609 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657612 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657614 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657617 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657619 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657621 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 02:25:52.663318 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657624 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657633 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657636 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657638 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657641 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657643 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657646 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657648 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657651 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657653 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657656 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657658 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657672 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657675 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657678 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657680 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657683 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657685 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657688 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657690 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 02:25:52.663803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657692 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 02:25:52.664428 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657695 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 02:25:52.664428 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:52.657698 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 02:25:52.664428 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.657702 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 02:25:52.664428 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.657798 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 30 02:25:52.664428 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.661224 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 30 02:25:52.664428 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.662922 2566 server.go:1019] "Starting client certificate rotation" Jul 30 02:25:52.664428 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.663032 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 02:25:52.664428 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.663083 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 02:25:52.666230 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.666215 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 02:25:52.667378 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.667362 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 02:25:52.674817 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.674801 2566 log.go:25] "Validated CRI v1 runtime API" Jul 30 02:25:52.677230 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.677215 2566 log.go:25] "Validated CRI v1 image API" Jul 30 02:25:52.678054 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.678041 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 30 02:25:52.680863 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.680843 2566 fs.go:135] Filesystem UUIDs: map[6abbca03-d180-4452-ad5b-cadf32a36d80:/dev/nvme0n1p4 74e9b849-a615-4ab4-9e66-b3984234a2fe:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 30 02:25:52.680934 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.680863 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 30 02:25:52.687818 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.687701 2566 manager.go:216] Machine: {Timestamp:2026-07-30 02:25:52.686236552 +0000 UTC m=+0.365040538 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099403 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2289dffe4d53cc27d71f89ed711510 SystemUUID:ec2289df-fe4d-53cc-27d7-1f89ed711510 BootID:e89cc7eb-76dc-4503-8b85-7598d7f3a9f1 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b9:f1:4a:1d:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b9:f1:4a:1d:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:e9:7d:dc:79:c5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 30 02:25:52.687818 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.687810 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 30 02:25:52.687975 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.687909 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 30 02:25:52.691329 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.691299 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 30 02:25:52.691458 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.691332 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-5.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 30 02:25:52.692205 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.692194 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 30 02:25:52.692247 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.692208 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 30 02:25:52.692247 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.692216 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 02:25:52.693069 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.693059 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 02:25:52.693768 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.693752 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 02:25:52.693927 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.693916 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 30 02:25:52.695678 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.695654 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 30 02:25:52.695798 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.695789 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 30 02:25:52.695833 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.695818 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 30 02:25:52.695861 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.695836 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 30 02:25:52.695861 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.695855 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 30 02:25:52.695914 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.695869 2566 kubelet.go:398] "Adding apiserver pod source" Jul 30 02:25:52.695914 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.695879 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 30 02:25:52.696426 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.696412 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 02:25:52.696490 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.696434 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 02:25:52.700477 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.700458 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 30 02:25:52.702347 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.702330 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 30 02:25:52.702434 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.702368 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 30 02:25:52.703882 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.703869 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 30 02:25:52.703948 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.703888 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 30 02:25:52.703948 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.703897 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 30 02:25:52.703948 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.703906 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 30 02:25:52.703948 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.703920 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 30 02:25:52.703948 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.703928 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 30 02:25:52.703948 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.703939 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 30 02:25:52.703948 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.703948 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 30 02:25:52.704171 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.703957 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 30 02:25:52.704171 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.703966 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 30 02:25:52.704171 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.703978 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 30 02:25:52.704171 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.703986 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 30 02:25:52.705727 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.705715 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 30 02:25:52.705778 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.705730 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 30 02:25:52.707024 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.706997 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 30 02:25:52.707024 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.706997 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-5.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 30 02:25:52.708792 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.708778 2566 server.go:1268] "Started kubelet" Jul 30 02:25:52.708871 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.708847 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 30 02:25:52.708997 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.708914 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 30 02:25:52.709051 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.709011 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 30 02:25:52.709560 ip-10-0-131-5 systemd[1]: Started Kubernetes Kubelet. Jul 30 02:25:52.710631 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.710612 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 30 02:25:52.711735 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.711719 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 30 02:25:52.715957 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.715938 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 30 02:25:52.718296 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.718273 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-5.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 30 02:25:52.719214 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.719195 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 30 02:25:52.719299 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.719226 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 30 02:25:52.719362 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.719291 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:52.719574 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.718414 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-5.ec2.internal.18c6f02706e09742 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-5.ec2.internal,UID:ip-10-0-131-5.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-5.ec2.internal,},FirstTimestamp:2026-07-30 02:25:52.708753218 +0000 UTC m=+0.387557203,LastTimestamp:2026-07-30 02:25:52.708753218 +0000 UTC m=+0.387557203,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-5.ec2.internal,}" Jul 30 02:25:52.720135 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.719859 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 30 02:25:52.720135 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.719884 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 30 02:25:52.720135 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.719952 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 30 02:25:52.720135 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.719980 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 30 02:25:52.720135 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.719992 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 30 02:25:52.720395 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.720334 2566 factory.go:55] Registering systemd factory Jul 30 02:25:52.720395 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.720348 2566 factory.go:223] Registration of the systemd container factory successfully Jul 30 02:25:52.721448 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.720729 2566 factory.go:153] Registering CRI-O factory Jul 30 02:25:52.721448 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.720747 2566 factory.go:223] Registration of the crio container factory successfully Jul 30 02:25:52.721448 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.720798 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 30 02:25:52.721448 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.720823 2566 factory.go:103] Registering Raw factory Jul 30 02:25:52.721448 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.720839 2566 manager.go:1196] Started watching for new ooms in manager Jul 30 02:25:52.721448 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.721252 2566 manager.go:349] Starting recovery of all containers Jul 30 02:25:52.730407 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.730367 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-5.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 30 02:25:52.730499 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.730449 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 30 02:25:52.733523 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.732764 2566 manager.go:354] Recovery completed Jul 30 02:25:52.735095 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.735076 2566 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 30 02:25:52.737516 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.737499 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rjpxn" Jul 30 02:25:52.738003 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.737992 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 02:25:52.739835 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.739794 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory" Jul 30 02:25:52.739835 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.739827 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 02:25:52.739935 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.739843 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID" Jul 30 02:25:52.740370 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.740356 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 30 02:25:52.740370 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.740370 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 30 02:25:52.740489 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.740387 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 30 02:25:52.740489 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.740427 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 02:25:52.741861 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.741776 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-5.ec2.internal.18c6f02708ba7293 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-5.ec2.internal,UID:ip-10-0-131-5.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-5.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-5.ec2.internal,},FirstTimestamp:2026-07-30 02:25:52.739807891 +0000 UTC m=+0.418611882,LastTimestamp:2026-07-30 02:25:52.739807891 +0000 UTC m=+0.418611882,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-5.ec2.internal,}" Jul 30 02:25:52.741961 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.741929 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory" Jul 30 02:25:52.741961 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.741956 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 02:25:52.742058 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.741965 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID" Jul 30 02:25:52.742058 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.741993 2566 policy_none.go:49] "None policy: Start" Jul 30 02:25:52.742058 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.742016 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 30 02:25:52.742058 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.742028 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 30 02:25:52.743207 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.743197 2566 policy_none.go:47] "Start" Jul 30 02:25:52.745447 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.745428 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rjpxn" Jul 30 02:25:52.750471 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.750398 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-5.ec2.internal.18c6f02708badec6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-5.ec2.internal,UID:ip-10-0-131-5.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-5.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-5.ec2.internal,},FirstTimestamp:2026-07-30 02:25:52.73983559 +0000 UTC m=+0.418639577,LastTimestamp:2026-07-30 02:25:52.73983559 +0000 UTC m=+0.418639577,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-5.ec2.internal,}" Jul 30 02:25:52.777330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.777189 2566 manager.go:337] "Starting Device Plugin manager" Jul 30 02:25:52.777396 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.777372 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 30 02:25:52.777396 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.777385 2566 server.go:87] "Starting device plugin registration server" Jul 30 02:25:52.777586 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.777575 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 30 02:25:52.777624 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.777586 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 30 02:25:52.777731 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.777718 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 30 02:25:52.777810 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.777799 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 30 02:25:52.777857 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.777810 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 30 02:25:52.778299 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.778269 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 30 02:25:52.778356 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.778324 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:52.844963 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.844929 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 30 02:25:52.846239 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.846220 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 30 02:25:52.846334 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.846262 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 30 02:25:52.846334 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.846288 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 30 02:25:52.846427 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.846330 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 30 02:25:52.848780 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.848760 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 02:25:52.877722 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.877689 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 02:25:52.879208 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.879193 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory" Jul 30 02:25:52.879268 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.879224 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 02:25:52.879268 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.879235 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID" Jul 30 02:25:52.879268 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.879258 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-5.ec2.internal" Jul 30 02:25:52.888656 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.888613 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-5.ec2.internal" Jul 30 02:25:52.888656 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.888635 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-5.ec2.internal\": node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:52.915913 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.915888 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:52.947377 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.947354 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal"] Jul 30 02:25:52.947465 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.947421 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 02:25:52.948251 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.948235 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory" Jul 30 02:25:52.948339 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.948268 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 02:25:52.948339 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.948283 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID" Jul 30 02:25:52.949623 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.949608 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 02:25:52.949760 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.949745 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" Jul 30 02:25:52.949829 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.949774 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 02:25:52.950335 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.950317 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory" Jul 30 02:25:52.950403 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.950350 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 02:25:52.950403 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.950361 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID" Jul 30 02:25:52.950486 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.950320 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory" Jul 30 02:25:52.950486 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.950427 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 02:25:52.950486 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.950441 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID" Jul 30 02:25:52.951605 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.951588 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal" Jul 30 02:25:52.951659 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.951616 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 02:25:52.952306 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.952278 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory" Jul 30 02:25:52.952306 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.952305 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 02:25:52.952455 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:52.952319 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID" Jul 30 02:25:52.977409 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.977388 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-5.ec2.internal\" not found" node="ip-10-0-131-5.ec2.internal" Jul 30 02:25:52.982067 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:52.982052 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-5.ec2.internal\" not found" node="ip-10-0-131-5.ec2.internal" Jul 30 02:25:53.016996 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:53.016977 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:53.021584 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.021563 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3409f2fa339f40a2f8e8edf5c4c6e45d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal\" (UID: \"3409f2fa339f40a2f8e8edf5c4c6e45d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" Jul 30 02:25:53.021646 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.021594 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3409f2fa339f40a2f8e8edf5c4c6e45d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal\" (UID: \"3409f2fa339f40a2f8e8edf5c4c6e45d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" Jul 30 02:25:53.021646 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.021626 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a52f7c3fa8fcd9dbf280379edf7b0272-config\") pod \"kube-apiserver-proxy-ip-10-0-131-5.ec2.internal\" (UID: \"a52f7c3fa8fcd9dbf280379edf7b0272\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal" Jul 30 02:25:53.117419 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:53.117388 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:53.122105 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.122089 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3409f2fa339f40a2f8e8edf5c4c6e45d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal\" (UID: \"3409f2fa339f40a2f8e8edf5c4c6e45d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" Jul 30 02:25:53.122160 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.122114 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3409f2fa339f40a2f8e8edf5c4c6e45d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal\" (UID: \"3409f2fa339f40a2f8e8edf5c4c6e45d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" Jul 30 02:25:53.122160 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.122140 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a52f7c3fa8fcd9dbf280379edf7b0272-config\") pod \"kube-apiserver-proxy-ip-10-0-131-5.ec2.internal\" (UID: \"a52f7c3fa8fcd9dbf280379edf7b0272\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal" Jul 30 02:25:53.122267 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.122180 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a52f7c3fa8fcd9dbf280379edf7b0272-config\") pod \"kube-apiserver-proxy-ip-10-0-131-5.ec2.internal\" (UID: \"a52f7c3fa8fcd9dbf280379edf7b0272\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal" Jul 30 02:25:53.122267 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.122192 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3409f2fa339f40a2f8e8edf5c4c6e45d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal\" (UID: \"3409f2fa339f40a2f8e8edf5c4c6e45d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" Jul 30 02:25:53.122267 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.122229 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3409f2fa339f40a2f8e8edf5c4c6e45d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal\" (UID: \"3409f2fa339f40a2f8e8edf5c4c6e45d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" Jul 30 02:25:53.218791 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:53.218737 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:53.284146 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.284118 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" Jul 30 02:25:53.285311 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.285290 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal" Jul 30 02:25:53.319057 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:53.319033 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:53.422956 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:53.422917 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:53.523900 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:53.523837 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:53.531465 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.531448 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 02:25:53.625554 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:53.625527 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:53.663616 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.663594 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 30 02:25:53.664052 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.663758 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 30 02:25:53.664052 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.663756 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 30 02:25:53.726141 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:53.726118 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:53.749076 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.749050 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 02:20:53 +0000 UTC" deadline="2028-05-02 04:39:47.710964574 +0000 UTC" Jul 30 02:25:53.749076 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.749074 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15410h13m53.961893508s" Jul 30 02:25:53.826961 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:53.826905 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:53.857966 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:53.857937 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3409f2fa339f40a2f8e8edf5c4c6e45d.slice/crio-6453cac701d1544a117afaf282d1c42fe4a7136360426e49bd45a76ea4fde46c WatchSource:0}: Error finding container 6453cac701d1544a117afaf282d1c42fe4a7136360426e49bd45a76ea4fde46c: Status 404 returned error can't find the container with id 6453cac701d1544a117afaf282d1c42fe4a7136360426e49bd45a76ea4fde46c Jul 30 02:25:53.863179 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:53.862999 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52f7c3fa8fcd9dbf280379edf7b0272.slice/crio-4526a4186d34434a3864746167b1f56ca477cf596fb81cc9c408c24f24eaf9d9 WatchSource:0}: Error finding container 4526a4186d34434a3864746167b1f56ca477cf596fb81cc9c408c24f24eaf9d9: Status 404 returned error can't find the container with id 4526a4186d34434a3864746167b1f56ca477cf596fb81cc9c408c24f24eaf9d9 Jul 30 02:25:53.866593 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.866577 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 02:25:53.909674 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.909640 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 02:25:53.910710 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.910685 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientMemory" Jul 30 02:25:53.910786 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.910724 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 02:25:53.910786 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:53.910735 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeHasSufficientPID" Jul 30 02:25:53.927727 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:53.927709 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:54.028430 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:54.028397 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-5.ec2.internal\" not found" Jul 30 02:25:54.065341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.065324 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 02:25:54.123757 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.123696 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" Jul 30 02:25:54.131494 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.131478 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 02:25:54.136897 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.136883 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 02:25:54.138179 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.138165 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal" Jul 30 02:25:54.145394 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.145379 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 02:25:54.698214 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.698174 2566 apiserver.go:52] "Watching apiserver" Jul 30 02:25:54.705243 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.705222 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 30 02:25:54.705598 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.705576 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-9l7r4","kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal","openshift-dns/node-resolver-84dj8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal","openshift-multus/multus-additional-cni-plugins-fpm66","openshift-multus/multus-gmdnj","openshift-multus/network-metrics-daemon-rx95s","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm","openshift-cluster-node-tuning-operator/tuned-8zfm4","openshift-image-registry/node-ca-tsrrd","openshift-network-diagnostics/network-check-target-z6stl","openshift-network-operator/iptables-alerter-j4rqd","openshift-ovn-kubernetes/ovnkube-node-pkwg7","kube-system/global-pull-secret-syncer-dp2xm"] Jul 30 02:25:54.708025 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.708004 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tsrrd" Jul 30 02:25:54.710496 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.710476 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.712167 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.712149 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-84dj8" Jul 30 02:25:54.712288 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.712269 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 30 02:25:54.712526 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.712511 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tlvkt\"" Jul 30 02:25:54.712579 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.712555 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 30 02:25:54.712987 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.712962 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 30 02:25:54.713374 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.713347 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9knf7\"" Jul 30 02:25:54.713488 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.713427 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 30 02:25:54.713488 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.713477 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 30 02:25:54.713603 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.713481 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 30 02:25:54.713603 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.713571 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 30 02:25:54.713603 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.713582 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 30 02:25:54.713603 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.713574 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 30 02:25:54.713957 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.713936 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:54.714041 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:54.714016 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:25:54.715505 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.715488 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wp249\"" Jul 30 02:25:54.716000 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.715975 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.716172 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.716155 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 30 02:25:54.716265 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.715976 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 30 02:25:54.717052 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.717032 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.718167 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.718148 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.718567 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.718548 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 30 02:25:54.718663 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.718548 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8cwr6\"" Jul 30 02:25:54.718663 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.718551 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 30 02:25:54.718774 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.718747 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 30 02:25:54.718979 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.718964 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 30 02:25:54.719049 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.719014 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 30 02:25:54.719378 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.719353 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qnnvm\"" Jul 30 02:25:54.719475 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.719394 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 30 02:25:54.719594 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.719575 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:25:54.719648 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:54.719634 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:25:54.720552 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.720534 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 30 02:25:54.720622 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.720611 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 30 02:25:54.720980 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.720955 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 30 02:25:54.721223 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.721206 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.721332 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.721209 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fp77g\"" Jul 30 02:25:54.722595 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.722580 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9l7r4" Jul 30 02:25:54.723869 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.723850 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 30 02:25:54.723967 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.723918 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q28cn\"" Jul 30 02:25:54.724087 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.723969 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:25:54.724087 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.723991 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 30 02:25:54.724087 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.723859 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 30 02:25:54.724087 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:54.724042 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:25:54.725059 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.725042 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 30 02:25:54.725118 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.725074 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 30 02:25:54.726472 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.726334 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4gwwr\"" Jul 30 02:25:54.726553 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.726535 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j4rqd" Jul 30 02:25:54.729246 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.729228 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 30 02:25:54.729605 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.729587 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 30 02:25:54.729605 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.729595 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n4mnt\"" Jul 30 02:25:54.729768 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.729590 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 30 02:25:54.731594 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731573 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-etc-selinux\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.731655 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731605 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-run-ovn\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.731655 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731633 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w67x\" (UniqueName: \"kubernetes.io/projected/78492ecc-d43f-4513-a1f7-0470faef225f-kube-api-access-7w67x\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.731770 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731658 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-sysctl-conf\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.731770 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731679 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-host\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.731770 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731712 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.731770 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731728 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-ovn-node-metrics-cert\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.731952 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731767 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/71aaec03-629a-427a-9207-6f58e2540c54-os-release\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.731952 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731804 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-multus-cni-dir\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.731952 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731833 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-sys\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.731952 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731861 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzdkz\" (UniqueName: \"kubernetes.io/projected/2a06f56e-3de8-45af-b3aa-da354ff4e895-kube-api-access-kzdkz\") pod \"node-resolver-84dj8\" (UID: \"2a06f56e-3de8-45af-b3aa-da354ff4e895\") " pod="openshift-dns/node-resolver-84dj8" Jul 30 02:25:54.731952 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731889 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71aaec03-629a-427a-9207-6f58e2540c54-system-cni-dir\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.731952 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71aaec03-629a-427a-9207-6f58e2540c54-cni-binary-copy\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.731952 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731944 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-multus-socket-dir-parent\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.732330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.731971 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-run-netns\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.732330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732039 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-etc-kubernetes\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.732330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732066 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-tuned\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.732330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732127 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9chq\" (UniqueName: \"kubernetes.io/projected/3b6d7d15-149c-4ae8-b6d8-741839934430-kube-api-access-p9chq\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.732330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732168 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-run-systemd\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.732330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732227 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-device-dir\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.732330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732252 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-log-socket\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.732330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732278 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-ovnkube-script-lib\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.732330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732300 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s8lz\" (UniqueName: \"kubernetes.io/projected/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-kube-api-access-4s8lz\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.732330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732318 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a06f56e-3de8-45af-b3aa-da354ff4e895-tmp-dir\") pod \"node-resolver-84dj8\" (UID: \"2a06f56e-3de8-45af-b3aa-da354ff4e895\") " pod="openshift-dns/node-resolver-84dj8" Jul 30 02:25:54.732706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732342 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1aca484e-f11d-45ae-acbb-e227a511b00c-kubelet-config\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:54.732706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732373 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-sysconfig\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.732706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732401 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-run\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.732706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732430 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5ssv\" (UniqueName: \"kubernetes.io/projected/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-kube-api-access-l5ssv\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.732706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732455 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-node-log\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.732706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732495 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-var-lib-cni-bin\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.732706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732523 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-sysctl-d\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.732706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732555 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjb7k\" (UniqueName: \"kubernetes.io/projected/db23d664-ec37-4206-b1f3-ac52ecec65dc-kube-api-access-fjb7k\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:25:54.732706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732588 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-registration-dir\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.732706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732613 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-etc-openvswitch\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.732706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732631 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-var-lib-kubelet\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.732706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732657 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-hostroot\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.732706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732700 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1aca484e-f11d-45ae-acbb-e227a511b00c-dbus\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732726 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-run-ovn-kubernetes\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732758 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-cni-netd\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732789 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71aaec03-629a-427a-9207-6f58e2540c54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732813 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-modprobe-d\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rscjk\" (UniqueName: \"kubernetes.io/projected/735cd8d6-2250-4b53-abef-118129c12350-kube-api-access-rscjk\") pod \"node-ca-tsrrd\" (UID: \"735cd8d6-2250-4b53-abef-118129c12350\") " pod="openshift-image-registry/node-ca-tsrrd" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732885 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-kubelet\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732911 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-system-cni-dir\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732948 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-os-release\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.732991 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/78492ecc-d43f-4513-a1f7-0470faef225f-multus-daemon-config\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733021 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/735cd8d6-2250-4b53-abef-118129c12350-host\") pod \"node-ca-tsrrd\" (UID: \"735cd8d6-2250-4b53-abef-118129c12350\") " pod="openshift-image-registry/node-ca-tsrrd" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733054 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71aaec03-629a-427a-9207-6f58e2540c54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733084 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-run-k8s-cni-cncf-io\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733113 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-var-lib-cni-multus\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733137 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-kubernetes\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733161 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:54.733254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733193 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-slash\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733231 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz4mk\" (UniqueName: \"kubernetes.io/projected/71aaec03-629a-427a-9207-6f58e2540c54-kube-api-access-gz4mk\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733275 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-var-lib-kubelet\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733325 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-tmp\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733357 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/735cd8d6-2250-4b53-abef-118129c12350-serviceca\") pod \"node-ca-tsrrd\" (UID: \"735cd8d6-2250-4b53-abef-118129c12350\") " pod="openshift-image-registry/node-ca-tsrrd" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733401 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733424 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-sys-fs\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733447 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-systemd-units\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733470 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a06f56e-3de8-45af-b3aa-da354ff4e895-hosts-file\") pod \"node-resolver-84dj8\" (UID: \"2a06f56e-3de8-45af-b3aa-da354ff4e895\") " pod="openshift-dns/node-resolver-84dj8" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733498 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-var-lib-openvswitch\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733521 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-ovnkube-config\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733547 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71aaec03-629a-427a-9207-6f58e2540c54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733575 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/78492ecc-d43f-4513-a1f7-0470faef225f-cni-binary-copy\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733600 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-run-multus-certs\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733622 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733655 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-systemd\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.733998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733720 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-lib-modules\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.734495 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733742 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-run-openvswitch\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.734495 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733757 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-cni-bin\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.734495 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733771 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-multus-conf-dir\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.734495 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733846 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-run-netns\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.734495 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733898 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71aaec03-629a-427a-9207-6f58e2540c54-cnibin\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.734495 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733926 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-socket-dir\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.734495 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733941 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-env-overrides\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.734495 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.733985 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-cnibin\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.735342 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.735325 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 02:25:54.753371 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.753354 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4z2pp" Jul 30 02:25:54.763071 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.763048 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4z2pp" Jul 30 02:25:54.822171 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.822017 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 30 02:25:54.825642 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.825621 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 02:25:54.834819 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.834786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-sysctl-conf\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.834923 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.834822 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-host\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.834923 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.834851 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.834923 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.834878 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-ovn-node-metrics-cert\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.834923 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.834905 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/71aaec03-629a-427a-9207-6f58e2540c54-os-release\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.835151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.834927 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-multus-cni-dir\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.835151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.834951 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-sys\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.835151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.834977 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzdkz\" (UniqueName: \"kubernetes.io/projected/2a06f56e-3de8-45af-b3aa-da354ff4e895-kube-api-access-kzdkz\") pod \"node-resolver-84dj8\" (UID: \"2a06f56e-3de8-45af-b3aa-da354ff4e895\") " pod="openshift-dns/node-resolver-84dj8" Jul 30 02:25:54.835151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.834994 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-sysctl-conf\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.835151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835006 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71aaec03-629a-427a-9207-6f58e2540c54-system-cni-dir\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.835151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835035 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71aaec03-629a-427a-9207-6f58e2540c54-cni-binary-copy\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.835151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835060 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-multus-socket-dir-parent\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.835151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835073 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/71aaec03-629a-427a-9207-6f58e2540c54-os-release\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.835151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835089 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-run-netns\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.835151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835115 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-etc-kubernetes\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.835151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835121 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-host\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.835151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835146 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfg5f\" (UniqueName: \"kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f\") pod \"network-check-target-z6stl\" (UID: \"32d47ce4-b22b-4416-b81d-3d12743dbdad\") " pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835159 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835173 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-tuned\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835212 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9chq\" (UniqueName: \"kubernetes.io/projected/3b6d7d15-149c-4ae8-b6d8-741839934430-kube-api-access-p9chq\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835240 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-run-systemd\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835267 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-device-dir\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835292 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-log-socket\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835325 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-ovnkube-script-lib\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835357 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s8lz\" (UniqueName: \"kubernetes.io/projected/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-kube-api-access-4s8lz\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835385 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52whj\" (UniqueName: \"kubernetes.io/projected/95cdb02e-32f7-442b-91da-05b7e9755270-kube-api-access-52whj\") pod \"iptables-alerter-j4rqd\" (UID: \"95cdb02e-32f7-442b-91da-05b7e9755270\") " pod="openshift-network-operator/iptables-alerter-j4rqd" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835411 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a06f56e-3de8-45af-b3aa-da354ff4e895-tmp-dir\") pod \"node-resolver-84dj8\" (UID: \"2a06f56e-3de8-45af-b3aa-da354ff4e895\") " pod="openshift-dns/node-resolver-84dj8" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835437 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1aca484e-f11d-45ae-acbb-e227a511b00c-kubelet-config\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835465 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-sysconfig\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835502 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-run\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.835761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835504 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835798 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-run-netns\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835934 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-multus-cni-dir\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.835979 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-sys\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836021 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5ssv\" (UniqueName: \"kubernetes.io/projected/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-kube-api-access-l5ssv\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836049 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-node-log\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836077 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-var-lib-cni-bin\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836106 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-sysctl-d\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836132 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjb7k\" (UniqueName: \"kubernetes.io/projected/db23d664-ec37-4206-b1f3-ac52ecec65dc-kube-api-access-fjb7k\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836157 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-registration-dir\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836184 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-etc-openvswitch\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836209 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-var-lib-kubelet\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836235 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-hostroot\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836240 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-ovnkube-script-lib\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836265 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95cdb02e-32f7-442b-91da-05b7e9755270-host-slash\") pod \"iptables-alerter-j4rqd\" (UID: \"95cdb02e-32f7-442b-91da-05b7e9755270\") " pod="openshift-network-operator/iptables-alerter-j4rqd" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836296 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1aca484e-f11d-45ae-acbb-e227a511b00c-dbus\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836390 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-run-ovn-kubernetes\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836393 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-hostroot\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.836480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836436 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-run-ovn-kubernetes\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836467 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-cni-netd\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836495 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71aaec03-629a-427a-9207-6f58e2540c54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836524 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9f09be67-5de5-4dc5-927e-9b1536d384e9-konnectivity-ca\") pod \"konnectivity-agent-9l7r4\" (UID: \"9f09be67-5de5-4dc5-927e-9b1536d384e9\") " pod="kube-system/konnectivity-agent-9l7r4" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836532 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-var-lib-kubelet\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836554 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-modprobe-d\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836584 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rscjk\" (UniqueName: \"kubernetes.io/projected/735cd8d6-2250-4b53-abef-118129c12350-kube-api-access-rscjk\") pod \"node-ca-tsrrd\" (UID: \"735cd8d6-2250-4b53-abef-118129c12350\") " pod="openshift-image-registry/node-ca-tsrrd" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836610 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-kubelet\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836638 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-system-cni-dir\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-os-release\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836692 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/78492ecc-d43f-4513-a1f7-0470faef225f-multus-daemon-config\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836716 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/735cd8d6-2250-4b53-abef-118129c12350-host\") pod \"node-ca-tsrrd\" (UID: \"735cd8d6-2250-4b53-abef-118129c12350\") " pod="openshift-image-registry/node-ca-tsrrd" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836756 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71aaec03-629a-427a-9207-6f58e2540c54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836785 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-run-k8s-cni-cncf-io\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836809 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1aca484e-f11d-45ae-acbb-e227a511b00c-dbus\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836811 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-var-lib-cni-multus\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836852 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-var-lib-cni-multus\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.837484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836859 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-kubernetes\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836891 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836967 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-run-systemd\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837100 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-etc-openvswitch\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837154 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-cni-netd\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836468 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-etc-kubernetes\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837269 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a06f56e-3de8-45af-b3aa-da354ff4e895-tmp-dir\") pod \"node-resolver-84dj8\" (UID: \"2a06f56e-3de8-45af-b3aa-da354ff4e895\") " pod="openshift-dns/node-resolver-84dj8" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837310 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71aaec03-629a-427a-9207-6f58e2540c54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837368 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-device-dir\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837418 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-sysconfig\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837424 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-modprobe-d\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837462 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1aca484e-f11d-45ae-acbb-e227a511b00c-kubelet-config\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837498 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-slash\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837528 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz4mk\" (UniqueName: \"kubernetes.io/projected/71aaec03-629a-427a-9207-6f58e2540c54-kube-api-access-gz4mk\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837559 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-var-lib-kubelet\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837585 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-tmp\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837613 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/735cd8d6-2250-4b53-abef-118129c12350-serviceca\") pod \"node-ca-tsrrd\" (UID: \"735cd8d6-2250-4b53-abef-118129c12350\") " pod="openshift-image-registry/node-ca-tsrrd" Jul 30 02:25:54.838413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837638 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.837686 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-kubelet\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838317 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71aaec03-629a-427a-9207-6f58e2540c54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.836755 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-registration-dir\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838340 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/78492ecc-d43f-4513-a1f7-0470faef225f-multus-daemon-config\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838396 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-run-k8s-cni-cncf-io\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:54.838421 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838431 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-sys-fs\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838463 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-systemd-units\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838493 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/735cd8d6-2250-4b53-abef-118129c12350-serviceca\") pod \"node-ca-tsrrd\" (UID: \"735cd8d6-2250-4b53-abef-118129c12350\") " pod="openshift-image-registry/node-ca-tsrrd" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:54.838516 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret podName:1aca484e-f11d-45ae-acbb-e227a511b00c nodeName:}" failed. No retries permitted until 2026-07-30 02:25:55.338473289 +0000 UTC m=+3.017277282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret") pod "global-pull-secret-syncer-dp2xm" (UID: "1aca484e-f11d-45ae-acbb-e227a511b00c") : object "kube-system"/"original-pull-secret" not registered Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838524 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-systemd-units\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838550 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-slash\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838563 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a06f56e-3de8-45af-b3aa-da354ff4e895-hosts-file\") pod \"node-resolver-84dj8\" (UID: \"2a06f56e-3de8-45af-b3aa-da354ff4e895\") " pod="openshift-dns/node-resolver-84dj8" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838582 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-var-lib-kubelet\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838596 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-var-lib-openvswitch\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838621 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-var-lib-cni-bin\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.839326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838627 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-ovnkube-config\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838660 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71aaec03-629a-427a-9207-6f58e2540c54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838691 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/78492ecc-d43f-4513-a1f7-0470faef225f-cni-binary-copy\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838706 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-sysctl-d\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838780 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-log-socket\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838873 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-run\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838868 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-sys-fs\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838916 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a06f56e-3de8-45af-b3aa-da354ff4e895-hosts-file\") pod \"node-resolver-84dj8\" (UID: \"2a06f56e-3de8-45af-b3aa-da354ff4e895\") " pod="openshift-dns/node-resolver-84dj8" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838931 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-system-cni-dir\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838946 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-os-release\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838955 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-var-lib-openvswitch\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838972 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71aaec03-629a-427a-9207-6f58e2540c54-system-cni-dir\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838991 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-kubernetes\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.838992 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-node-log\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839051 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-multus-socket-dir-parent\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839162 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839277 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/735cd8d6-2250-4b53-abef-118129c12350-host\") pod \"node-ca-tsrrd\" (UID: \"735cd8d6-2250-4b53-abef-118129c12350\") " pod="openshift-image-registry/node-ca-tsrrd" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839532 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71aaec03-629a-427a-9207-6f58e2540c54-cni-binary-copy\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.840341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839539 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/78492ecc-d43f-4513-a1f7-0470faef225f-cni-binary-copy\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839588 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-run-multus-certs\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839618 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839647 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-systemd\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839672 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-lib-modules\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-run-openvswitch\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839717 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-cni-bin\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839764 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-multus-conf-dir\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839768 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-ovn-node-metrics-cert\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839791 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-run-netns\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839816 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71aaec03-629a-427a-9207-6f58e2540c54-cnibin\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839843 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/95cdb02e-32f7-442b-91da-05b7e9755270-iptables-alerter-script\") pod \"iptables-alerter-j4rqd\" (UID: \"95cdb02e-32f7-442b-91da-05b7e9755270\") " pod="openshift-network-operator/iptables-alerter-j4rqd" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839869 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-socket-dir\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839894 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-env-overrides\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839920 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-cnibin\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839937 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-tuned\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839945 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-etc-selinux\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.841318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839950 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-cni-bin\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840012 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-host-run-multus-certs\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840023 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71aaec03-629a-427a-9207-6f58e2540c54-cnibin\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840051 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-multus-conf-dir\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840084 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-host-run-netns\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840098 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-etc-systemd\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840191 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-lib-modules\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:54.840194 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:54.840274 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs podName:db23d664-ec37-4206-b1f3-ac52ecec65dc nodeName:}" failed. No retries permitted until 2026-07-30 02:25:55.340256527 +0000 UTC m=+3.019060508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs") pod "network-metrics-daemon-rx95s" (UID: "db23d664-ec37-4206-b1f3-ac52ecec65dc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840309 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-run-openvswitch\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840361 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/78492ecc-d43f-4513-a1f7-0470faef225f-cnibin\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840436 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-etc-selinux\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.839972 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-run-ovn\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840487 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w67x\" (UniqueName: \"kubernetes.io/projected/78492ecc-d43f-4513-a1f7-0470faef225f-kube-api-access-7w67x\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840514 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9f09be67-5de5-4dc5-927e-9b1536d384e9-agent-certs\") pod \"konnectivity-agent-9l7r4\" (UID: \"9f09be67-5de5-4dc5-927e-9b1536d384e9\") " pod="kube-system/konnectivity-agent-9l7r4" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840539 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3b6d7d15-149c-4ae8-b6d8-741839934430-socket-dir\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840587 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-run-ovn\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.842291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.840601 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-env-overrides\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.843263 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.841085 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-ovnkube-config\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.843263 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.841278 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71aaec03-629a-427a-9207-6f58e2540c54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.844453 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.844428 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-tmp\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.845419 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.845395 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9chq\" (UniqueName: \"kubernetes.io/projected/3b6d7d15-149c-4ae8-b6d8-741839934430-kube-api-access-p9chq\") pod \"aws-ebs-csi-driver-node-s5xcm\" (UID: \"3b6d7d15-149c-4ae8-b6d8-741839934430\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:54.845872 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.845850 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjb7k\" (UniqueName: \"kubernetes.io/projected/db23d664-ec37-4206-b1f3-ac52ecec65dc-kube-api-access-fjb7k\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:25:54.849654 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.849532 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s8lz\" (UniqueName: \"kubernetes.io/projected/f62a99b1-985d-4f2a-b516-05e2aa1b6bee-kube-api-access-4s8lz\") pod \"ovnkube-node-pkwg7\" (UID: \"f62a99b1-985d-4f2a-b516-05e2aa1b6bee\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:54.849654 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.849640 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzdkz\" (UniqueName: \"kubernetes.io/projected/2a06f56e-3de8-45af-b3aa-da354ff4e895-kube-api-access-kzdkz\") pod \"node-resolver-84dj8\" (UID: \"2a06f56e-3de8-45af-b3aa-da354ff4e895\") " pod="openshift-dns/node-resolver-84dj8" Jul 30 02:25:54.849839 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.849655 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w67x\" (UniqueName: \"kubernetes.io/projected/78492ecc-d43f-4513-a1f7-0470faef225f-kube-api-access-7w67x\") pod \"multus-gmdnj\" (UID: \"78492ecc-d43f-4513-a1f7-0470faef225f\") " pod="openshift-multus/multus-gmdnj" Jul 30 02:25:54.850714 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.850626 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rscjk\" (UniqueName: \"kubernetes.io/projected/735cd8d6-2250-4b53-abef-118129c12350-kube-api-access-rscjk\") pod \"node-ca-tsrrd\" (UID: \"735cd8d6-2250-4b53-abef-118129c12350\") " pod="openshift-image-registry/node-ca-tsrrd" Jul 30 02:25:54.853909 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.853885 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz4mk\" (UniqueName: \"kubernetes.io/projected/71aaec03-629a-427a-9207-6f58e2540c54-kube-api-access-gz4mk\") pod \"multus-additional-cni-plugins-fpm66\" (UID: \"71aaec03-629a-427a-9207-6f58e2540c54\") " pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:54.854702 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.854672 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5ssv\" (UniqueName: \"kubernetes.io/projected/ca5823b8-3917-47f6-be47-a6a7c6fd9c02-kube-api-access-l5ssv\") pod \"tuned-8zfm4\" (UID: \"ca5823b8-3917-47f6-be47-a6a7c6fd9c02\") " pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:54.859030 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.857281 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal" event={"ID":"a52f7c3fa8fcd9dbf280379edf7b0272","Type":"ContainerStarted","Data":"4526a4186d34434a3864746167b1f56ca477cf596fb81cc9c408c24f24eaf9d9"} Jul 30 02:25:54.859606 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.859303 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" event={"ID":"3409f2fa339f40a2f8e8edf5c4c6e45d","Type":"ContainerStarted","Data":"6453cac701d1544a117afaf282d1c42fe4a7136360426e49bd45a76ea4fde46c"} Jul 30 02:25:54.941057 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.941026 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9f09be67-5de5-4dc5-927e-9b1536d384e9-konnectivity-ca\") pod \"konnectivity-agent-9l7r4\" (UID: \"9f09be67-5de5-4dc5-927e-9b1536d384e9\") " pod="kube-system/konnectivity-agent-9l7r4" Jul 30 02:25:54.941189 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.941122 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/95cdb02e-32f7-442b-91da-05b7e9755270-iptables-alerter-script\") pod \"iptables-alerter-j4rqd\" (UID: \"95cdb02e-32f7-442b-91da-05b7e9755270\") " pod="openshift-network-operator/iptables-alerter-j4rqd" Jul 30 02:25:54.941189 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.941151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9f09be67-5de5-4dc5-927e-9b1536d384e9-agent-certs\") pod \"konnectivity-agent-9l7r4\" (UID: \"9f09be67-5de5-4dc5-927e-9b1536d384e9\") " pod="kube-system/konnectivity-agent-9l7r4" Jul 30 02:25:54.941301 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.941198 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfg5f\" (UniqueName: \"kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f\") pod \"network-check-target-z6stl\" (UID: \"32d47ce4-b22b-4416-b81d-3d12743dbdad\") " pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:25:54.941301 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.941230 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52whj\" (UniqueName: \"kubernetes.io/projected/95cdb02e-32f7-442b-91da-05b7e9755270-kube-api-access-52whj\") pod \"iptables-alerter-j4rqd\" (UID: \"95cdb02e-32f7-442b-91da-05b7e9755270\") " pod="openshift-network-operator/iptables-alerter-j4rqd" Jul 30 02:25:54.941301 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.941264 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95cdb02e-32f7-442b-91da-05b7e9755270-host-slash\") pod \"iptables-alerter-j4rqd\" (UID: \"95cdb02e-32f7-442b-91da-05b7e9755270\") " pod="openshift-network-operator/iptables-alerter-j4rqd" Jul 30 02:25:54.941471 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.941326 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95cdb02e-32f7-442b-91da-05b7e9755270-host-slash\") pod \"iptables-alerter-j4rqd\" (UID: \"95cdb02e-32f7-442b-91da-05b7e9755270\") " pod="openshift-network-operator/iptables-alerter-j4rqd" Jul 30 02:25:54.941765 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.941741 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/95cdb02e-32f7-442b-91da-05b7e9755270-iptables-alerter-script\") pod \"iptables-alerter-j4rqd\" (UID: \"95cdb02e-32f7-442b-91da-05b7e9755270\") " pod="openshift-network-operator/iptables-alerter-j4rqd" Jul 30 02:25:54.941896 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.941769 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9f09be67-5de5-4dc5-927e-9b1536d384e9-konnectivity-ca\") pod \"konnectivity-agent-9l7r4\" (UID: \"9f09be67-5de5-4dc5-927e-9b1536d384e9\") " pod="kube-system/konnectivity-agent-9l7r4" Jul 30 02:25:54.944284 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.944261 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9f09be67-5de5-4dc5-927e-9b1536d384e9-agent-certs\") pod \"konnectivity-agent-9l7r4\" (UID: \"9f09be67-5de5-4dc5-927e-9b1536d384e9\") " pod="kube-system/konnectivity-agent-9l7r4" Jul 30 02:25:54.948325 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:54.948271 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 02:25:54.948325 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:54.948302 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 02:25:54.948325 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:54.948316 2566 projected.go:196] Error preparing data for projected volume kube-api-access-lfg5f for pod openshift-network-diagnostics/network-check-target-z6stl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:25:54.948527 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:54.948373 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f podName:32d47ce4-b22b-4416-b81d-3d12743dbdad nodeName:}" failed. No retries permitted until 2026-07-30 02:25:55.448356189 +0000 UTC m=+3.127160180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lfg5f" (UniqueName: "kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f") pod "network-check-target-z6stl" (UID: "32d47ce4-b22b-4416-b81d-3d12743dbdad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:25:54.951857 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:54.951834 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52whj\" (UniqueName: \"kubernetes.io/projected/95cdb02e-32f7-442b-91da-05b7e9755270-kube-api-access-52whj\") pod \"iptables-alerter-j4rqd\" (UID: \"95cdb02e-32f7-442b-91da-05b7e9755270\") " pod="openshift-network-operator/iptables-alerter-j4rqd" Jul 30 02:25:55.019428 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.019392 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tsrrd" Jul 30 02:25:55.027648 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.027475 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:25:55.030329 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:55.030299 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod735cd8d6_2250_4b53_abef_118129c12350.slice/crio-e0d3e1cf6864d801519487f029e5641118a96a8bcba44361f0b86223154012a8 WatchSource:0}: Error finding container e0d3e1cf6864d801519487f029e5641118a96a8bcba44361f0b86223154012a8: Status 404 returned error can't find the container with id e0d3e1cf6864d801519487f029e5641118a96a8bcba44361f0b86223154012a8 Jul 30 02:25:55.037467 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.037303 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-84dj8" Jul 30 02:25:55.041593 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:55.041556 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf62a99b1_985d_4f2a_b516_05e2aa1b6bee.slice/crio-2c404da161a8735b16365b16aea87c25e04e9cc72fb56215ccf011484ac5fffc WatchSource:0}: Error finding container 2c404da161a8735b16365b16aea87c25e04e9cc72fb56215ccf011484ac5fffc: Status 404 returned error can't find the container with id 2c404da161a8735b16365b16aea87c25e04e9cc72fb56215ccf011484ac5fffc Jul 30 02:25:55.042858 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.042671 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fpm66" Jul 30 02:25:55.049476 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.049454 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gmdnj" Jul 30 02:25:55.055641 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:55.055616 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71aaec03_629a_427a_9207_6f58e2540c54.slice/crio-e922a9794958c1912b4886760cbe28f835e212a69166006df41f7d7d83f7d693 WatchSource:0}: Error finding container e922a9794958c1912b4886760cbe28f835e212a69166006df41f7d7d83f7d693: Status 404 returned error can't find the container with id e922a9794958c1912b4886760cbe28f835e212a69166006df41f7d7d83f7d693 Jul 30 02:25:55.056958 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:55.056889 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a06f56e_3de8_45af_b3aa_da354ff4e895.slice/crio-3ac03dee7109c76edd72f16bbbdaf918e8938b1bf91f958edab8e0927890f66c WatchSource:0}: Error finding container 3ac03dee7109c76edd72f16bbbdaf918e8938b1bf91f958edab8e0927890f66c: Status 404 returned error can't find the container with id 3ac03dee7109c76edd72f16bbbdaf918e8938b1bf91f958edab8e0927890f66c Jul 30 02:25:55.062084 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.062063 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" Jul 30 02:25:55.070738 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.070721 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" Jul 30 02:25:55.076651 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:55.076203 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca5823b8_3917_47f6_be47_a6a7c6fd9c02.slice/crio-72d7c23cfbb7df5e76dc66ca49655d37cfce40dd8e49615573d9c7c3cdfbe1f1 WatchSource:0}: Error finding container 72d7c23cfbb7df5e76dc66ca49655d37cfce40dd8e49615573d9c7c3cdfbe1f1: Status 404 returned error can't find the container with id 72d7c23cfbb7df5e76dc66ca49655d37cfce40dd8e49615573d9c7c3cdfbe1f1 Jul 30 02:25:55.076651 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.076296 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9l7r4" Jul 30 02:25:55.076959 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:55.076929 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78492ecc_d43f_4513_a1f7_0470faef225f.slice/crio-d710a015728c5366c843b83207b2671896992969e45d712095fba46ffe2c1035 WatchSource:0}: Error finding container d710a015728c5366c843b83207b2671896992969e45d712095fba46ffe2c1035: Status 404 returned error can't find the container with id d710a015728c5366c843b83207b2671896992969e45d712095fba46ffe2c1035 Jul 30 02:25:55.082799 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.082780 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j4rqd" Jul 30 02:25:55.083281 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:55.083259 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b6d7d15_149c_4ae8_b6d8_741839934430.slice/crio-58504185dbf11f6c0c82006351be05bd80074f7845eb3e1d0224bad789a5b8b5 WatchSource:0}: Error finding container 58504185dbf11f6c0c82006351be05bd80074f7845eb3e1d0224bad789a5b8b5: Status 404 returned error can't find the container with id 58504185dbf11f6c0c82006351be05bd80074f7845eb3e1d0224bad789a5b8b5 Jul 30 02:25:55.094450 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:55.094417 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95cdb02e_32f7_442b_91da_05b7e9755270.slice/crio-2750ed999ebc787bbe88729c5a0ca30a02c1e64a665e390ff605bf233a43c956 WatchSource:0}: Error finding container 2750ed999ebc787bbe88729c5a0ca30a02c1e64a665e390ff605bf233a43c956: Status 404 returned error can't find the container with id 2750ed999ebc787bbe88729c5a0ca30a02c1e64a665e390ff605bf233a43c956 Jul 30 02:25:55.095837 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:25:55.095813 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f09be67_5de5_4dc5_927e_9b1536d384e9.slice/crio-fcfd6df7738b1b95eed09520aadcd00b6f07996ce837928c27269a5fac6fe88e WatchSource:0}: Error finding container fcfd6df7738b1b95eed09520aadcd00b6f07996ce837928c27269a5fac6fe88e: Status 404 returned error can't find the container with id fcfd6df7738b1b95eed09520aadcd00b6f07996ce837928c27269a5fac6fe88e Jul 30 02:25:55.343223 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.343133 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:55.343223 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.343182 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:25:55.343456 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:55.343390 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 02:25:55.343502 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:55.343467 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret podName:1aca484e-f11d-45ae-acbb-e227a511b00c nodeName:}" failed. No retries permitted until 2026-07-30 02:25:56.343446514 +0000 UTC m=+4.022250489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret") pod "global-pull-secret-syncer-dp2xm" (UID: "1aca484e-f11d-45ae-acbb-e227a511b00c") : object "kube-system"/"original-pull-secret" not registered Jul 30 02:25:55.343549 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:55.343510 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:25:55.343588 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:55.343555 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs podName:db23d664-ec37-4206-b1f3-ac52ecec65dc nodeName:}" failed. No retries permitted until 2026-07-30 02:25:56.343540938 +0000 UTC m=+4.022344916 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs") pod "network-metrics-daemon-rx95s" (UID: "db23d664-ec37-4206-b1f3-ac52ecec65dc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:25:55.544099 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.544065 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfg5f\" (UniqueName: \"kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f\") pod \"network-check-target-z6stl\" (UID: \"32d47ce4-b22b-4416-b81d-3d12743dbdad\") " pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:25:55.544300 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:55.544262 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 02:25:55.544300 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:55.544290 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 02:25:55.544407 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:55.544306 2566 projected.go:196] Error preparing data for projected volume kube-api-access-lfg5f for pod openshift-network-diagnostics/network-check-target-z6stl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:25:55.544407 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:55.544376 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f podName:32d47ce4-b22b-4416-b81d-3d12743dbdad nodeName:}" failed. No retries permitted until 2026-07-30 02:25:56.544354951 +0000 UTC m=+4.223158941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lfg5f" (UniqueName: "kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f") pod "network-check-target-z6stl" (UID: "32d47ce4-b22b-4416-b81d-3d12743dbdad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:25:55.765325 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.765226 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 02:20:55 +0000 UTC" deadline="2028-03-16 17:35:35.711471311 +0000 UTC" Jul 30 02:25:55.765325 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.765260 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14295h9m39.946215834s" Jul 30 02:25:55.846854 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.846821 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:25:55.847019 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.846853 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:55.847019 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:55.846965 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:25:55.847145 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:55.847107 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:25:55.861644 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.861614 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9l7r4" event={"ID":"9f09be67-5de5-4dc5-927e-9b1536d384e9","Type":"ContainerStarted","Data":"fcfd6df7738b1b95eed09520aadcd00b6f07996ce837928c27269a5fac6fe88e"} Jul 30 02:25:55.863128 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.863100 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" event={"ID":"3b6d7d15-149c-4ae8-b6d8-741839934430","Type":"ContainerStarted","Data":"58504185dbf11f6c0c82006351be05bd80074f7845eb3e1d0224bad789a5b8b5"} Jul 30 02:25:55.864489 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.864461 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gmdnj" event={"ID":"78492ecc-d43f-4513-a1f7-0470faef225f","Type":"ContainerStarted","Data":"d710a015728c5366c843b83207b2671896992969e45d712095fba46ffe2c1035"} Jul 30 02:25:55.865784 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.865748 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" event={"ID":"ca5823b8-3917-47f6-be47-a6a7c6fd9c02","Type":"ContainerStarted","Data":"72d7c23cfbb7df5e76dc66ca49655d37cfce40dd8e49615573d9c7c3cdfbe1f1"} Jul 30 02:25:55.867256 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.867227 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fpm66" event={"ID":"71aaec03-629a-427a-9207-6f58e2540c54","Type":"ContainerStarted","Data":"e922a9794958c1912b4886760cbe28f835e212a69166006df41f7d7d83f7d693"} Jul 30 02:25:55.868710 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.868659 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-84dj8" event={"ID":"2a06f56e-3de8-45af-b3aa-da354ff4e895","Type":"ContainerStarted","Data":"3ac03dee7109c76edd72f16bbbdaf918e8938b1bf91f958edab8e0927890f66c"} Jul 30 02:25:55.870439 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.870400 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" event={"ID":"f62a99b1-985d-4f2a-b516-05e2aa1b6bee","Type":"ContainerStarted","Data":"2c404da161a8735b16365b16aea87c25e04e9cc72fb56215ccf011484ac5fffc"} Jul 30 02:25:55.871621 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.871601 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tsrrd" event={"ID":"735cd8d6-2250-4b53-abef-118129c12350","Type":"ContainerStarted","Data":"e0d3e1cf6864d801519487f029e5641118a96a8bcba44361f0b86223154012a8"} Jul 30 02:25:55.873080 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:55.873057 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j4rqd" event={"ID":"95cdb02e-32f7-442b-91da-05b7e9755270","Type":"ContainerStarted","Data":"2750ed999ebc787bbe88729c5a0ca30a02c1e64a665e390ff605bf233a43c956"} Jul 30 02:25:56.261129 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:56.261044 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 02:25:56.351303 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:56.351262 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:56.351505 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:56.351322 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:25:56.351576 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:56.351514 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:25:56.351633 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:56.351580 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs podName:db23d664-ec37-4206-b1f3-ac52ecec65dc nodeName:}" failed. No retries permitted until 2026-07-30 02:25:58.351560595 +0000 UTC m=+6.030364578 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs") pod "network-metrics-daemon-rx95s" (UID: "db23d664-ec37-4206-b1f3-ac52ecec65dc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:25:56.352284 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:56.351815 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 02:25:56.352284 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:56.351861 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret podName:1aca484e-f11d-45ae-acbb-e227a511b00c nodeName:}" failed. No retries permitted until 2026-07-30 02:25:58.351846658 +0000 UTC m=+6.030650633 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret") pod "global-pull-secret-syncer-dp2xm" (UID: "1aca484e-f11d-45ae-acbb-e227a511b00c") : object "kube-system"/"original-pull-secret" not registered Jul 30 02:25:56.553177 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:56.553045 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfg5f\" (UniqueName: \"kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f\") pod \"network-check-target-z6stl\" (UID: \"32d47ce4-b22b-4416-b81d-3d12743dbdad\") " pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:25:56.553414 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:56.553271 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 02:25:56.553414 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:56.553292 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 02:25:56.553414 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:56.553304 2566 projected.go:196] Error preparing data for projected volume kube-api-access-lfg5f for pod openshift-network-diagnostics/network-check-target-z6stl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:25:56.553414 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:56.553364 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f podName:32d47ce4-b22b-4416-b81d-3d12743dbdad nodeName:}" failed. No retries permitted until 2026-07-30 02:25:58.553342435 +0000 UTC m=+6.232146415 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lfg5f" (UniqueName: "kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f") pod "network-check-target-z6stl" (UID: "32d47ce4-b22b-4416-b81d-3d12743dbdad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:25:56.765629 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:56.765583 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 02:20:55 +0000 UTC" deadline="2028-01-13 21:17:25.98603562 +0000 UTC" Jul 30 02:25:56.765629 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:56.765627 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12786h51m29.220413864s" Jul 30 02:25:56.846612 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:56.846535 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:25:56.846843 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:56.846708 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:25:57.847584 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:57.847550 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:25:57.848131 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:57.847693 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:25:57.848131 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:57.847774 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:57.848131 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:57.847849 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:25:58.369720 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:58.369668 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:25:58.369915 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:58.369778 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:58.369915 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:58.369908 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 02:25:58.370030 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:58.369996 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret podName:1aca484e-f11d-45ae-acbb-e227a511b00c nodeName:}" failed. No retries permitted until 2026-07-30 02:26:02.369975983 +0000 UTC m=+10.048779961 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret") pod "global-pull-secret-syncer-dp2xm" (UID: "1aca484e-f11d-45ae-acbb-e227a511b00c") : object "kube-system"/"original-pull-secret" not registered Jul 30 02:25:58.370414 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:58.370395 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:25:58.370533 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:58.370449 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs podName:db23d664-ec37-4206-b1f3-ac52ecec65dc nodeName:}" failed. No retries permitted until 2026-07-30 02:26:02.370433719 +0000 UTC m=+10.049237698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs") pod "network-metrics-daemon-rx95s" (UID: "db23d664-ec37-4206-b1f3-ac52ecec65dc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:25:58.572050 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:58.571236 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfg5f\" (UniqueName: \"kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f\") pod \"network-check-target-z6stl\" (UID: \"32d47ce4-b22b-4416-b81d-3d12743dbdad\") " pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:25:58.572050 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:58.571435 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 02:25:58.572050 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:58.571460 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 02:25:58.572050 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:58.571474 2566 projected.go:196] Error preparing data for projected volume kube-api-access-lfg5f for pod openshift-network-diagnostics/network-check-target-z6stl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:25:58.572050 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:58.571532 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f podName:32d47ce4-b22b-4416-b81d-3d12743dbdad nodeName:}" failed. No retries permitted until 2026-07-30 02:26:02.571513575 +0000 UTC m=+10.250317553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lfg5f" (UniqueName: "kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f") pod "network-check-target-z6stl" (UID: "32d47ce4-b22b-4416-b81d-3d12743dbdad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:25:58.847274 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:58.846540 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:25:58.847274 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:58.846715 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:25:59.847076 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:59.846907 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:25:59.847076 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:59.847041 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:25:59.847652 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:25:59.847158 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:25:59.847652 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:25:59.847281 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:00.848312 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:00.847789 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:00.848312 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:00.847941 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:01.846560 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:01.846482 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:01.846560 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:01.846506 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:01.846840 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:01.846605 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:01.846840 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:01.846830 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:02.402648 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:02.402399 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:02.402648 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:02.402454 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:02.402648 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:02.402569 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 02:26:02.402648 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:02.402597 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:26:02.402648 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:02.402640 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret podName:1aca484e-f11d-45ae-acbb-e227a511b00c nodeName:}" failed. No retries permitted until 2026-07-30 02:26:10.402620781 +0000 UTC m=+18.081424768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret") pod "global-pull-secret-syncer-dp2xm" (UID: "1aca484e-f11d-45ae-acbb-e227a511b00c") : object "kube-system"/"original-pull-secret" not registered Jul 30 02:26:02.402648 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:02.402676 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs podName:db23d664-ec37-4206-b1f3-ac52ecec65dc nodeName:}" failed. No retries permitted until 2026-07-30 02:26:10.402650214 +0000 UTC m=+18.081454193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs") pod "network-metrics-daemon-rx95s" (UID: "db23d664-ec37-4206-b1f3-ac52ecec65dc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:26:02.604601 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:02.604559 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfg5f\" (UniqueName: \"kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f\") pod \"network-check-target-z6stl\" (UID: \"32d47ce4-b22b-4416-b81d-3d12743dbdad\") " pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:02.604807 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:02.604788 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 02:26:02.604876 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:02.604813 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 02:26:02.604876 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:02.604826 2566 projected.go:196] Error preparing data for projected volume kube-api-access-lfg5f for pod openshift-network-diagnostics/network-check-target-z6stl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:26:02.604970 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:02.604884 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f podName:32d47ce4-b22b-4416-b81d-3d12743dbdad nodeName:}" failed. No retries permitted until 2026-07-30 02:26:10.604865824 +0000 UTC m=+18.283669798 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lfg5f" (UniqueName: "kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f") pod "network-check-target-z6stl" (UID: "32d47ce4-b22b-4416-b81d-3d12743dbdad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:26:02.850728 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:02.850344 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:02.850728 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:02.850477 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:03.846863 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:03.846834 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:03.846863 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:03.846863 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:03.847333 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:03.846953 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:03.847333 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:03.847033 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:04.851156 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:04.851115 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:04.851620 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:04.851269 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:05.847233 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:05.847200 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:05.847233 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:05.847224 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:05.847442 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:05.847316 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:05.847442 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:05.847423 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:06.847850 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:06.847582 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:06.848304 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:06.847962 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:07.846837 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:07.846807 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:07.847022 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:07.846807 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:07.847022 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:07.846923 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:07.847022 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:07.847008 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:08.846917 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:08.846875 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:08.847349 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:08.847030 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:09.846762 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:09.846731 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:09.846973 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:09.846837 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:09.846973 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:09.846891 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:09.847322 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:09.846985 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:10.464803 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:10.464721 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:10.464803 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:10.464777 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:10.464987 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:10.464881 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:26:10.464987 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:10.464883 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 02:26:10.464987 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:10.464942 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs podName:db23d664-ec37-4206-b1f3-ac52ecec65dc nodeName:}" failed. No retries permitted until 2026-07-30 02:26:26.464923462 +0000 UTC m=+34.143727436 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs") pod "network-metrics-daemon-rx95s" (UID: "db23d664-ec37-4206-b1f3-ac52ecec65dc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:26:10.464987 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:10.464958 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret podName:1aca484e-f11d-45ae-acbb-e227a511b00c nodeName:}" failed. No retries permitted until 2026-07-30 02:26:26.464951158 +0000 UTC m=+34.143755130 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret") pod "global-pull-secret-syncer-dp2xm" (UID: "1aca484e-f11d-45ae-acbb-e227a511b00c") : object "kube-system"/"original-pull-secret" not registered Jul 30 02:26:10.666937 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:10.666899 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfg5f\" (UniqueName: \"kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f\") pod \"network-check-target-z6stl\" (UID: \"32d47ce4-b22b-4416-b81d-3d12743dbdad\") " pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:10.667084 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:10.667061 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 02:26:10.667150 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:10.667089 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 02:26:10.667150 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:10.667105 2566 projected.go:196] Error preparing data for projected volume kube-api-access-lfg5f for pod openshift-network-diagnostics/network-check-target-z6stl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:26:10.667220 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:10.667159 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f podName:32d47ce4-b22b-4416-b81d-3d12743dbdad nodeName:}" failed. No retries permitted until 2026-07-30 02:26:26.667142988 +0000 UTC m=+34.345946960 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lfg5f" (UniqueName: "kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f") pod "network-check-target-z6stl" (UID: "32d47ce4-b22b-4416-b81d-3d12743dbdad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:26:10.847352 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:10.847273 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:10.847790 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:10.847393 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:11.847080 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:11.847043 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:11.847246 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:11.847161 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:11.847246 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:11.847218 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:11.847370 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:11.847351 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:12.849394 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:12.849368 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:12.849815 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:12.849467 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:13.847427 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:13.847400 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:13.847594 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:13.847400 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:13.847594 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:13.847496 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:13.847594 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:13.847582 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:14.847687 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.847132 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:14.847687 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:14.847446 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:14.919097 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.919071 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" event={"ID":"3409f2fa339f40a2f8e8edf5c4c6e45d","Type":"ContainerStarted","Data":"5e9f9c205262828890a0a426bf98d0b22deffced67985ee9812488086b060253"} Jul 30 02:26:14.924857 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.924834 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9l7r4" event={"ID":"9f09be67-5de5-4dc5-927e-9b1536d384e9","Type":"ContainerStarted","Data":"4fb9aeb1788940ea98b74a4d745c2351be9e8ef1f5d7d25af8f09e4a786372af"} Jul 30 02:26:14.926063 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.926036 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" event={"ID":"3b6d7d15-149c-4ae8-b6d8-741839934430","Type":"ContainerStarted","Data":"52f2f3e01bd2bb406167088576ee06ed98cd84b9daceafb19f46a777b227d312"} Jul 30 02:26:14.927378 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.927356 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gmdnj" event={"ID":"78492ecc-d43f-4513-a1f7-0470faef225f","Type":"ContainerStarted","Data":"0a064e0d32462205c482614366237f114d23f2f44b945d603859b08b90ae5520"} Jul 30 02:26:14.928774 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.928728 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" event={"ID":"ca5823b8-3917-47f6-be47-a6a7c6fd9c02","Type":"ContainerStarted","Data":"1075737d61e9c4a4a1734c794bf0df224f32f1967312135e752e5ee6c39281ae"} Jul 30 02:26:14.930078 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.930028 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fpm66" event={"ID":"71aaec03-629a-427a-9207-6f58e2540c54","Type":"ContainerStarted","Data":"a234f1b1739126264c07de54f2896ae0835d7612f2bfe29316432e4e20de650b"} Jul 30 02:26:14.931107 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.931088 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-84dj8" event={"ID":"2a06f56e-3de8-45af-b3aa-da354ff4e895","Type":"ContainerStarted","Data":"c161bf75426cd394a473a7289ceb545fcec7307a753b49066026efa62266aa0d"} Jul 30 02:26:14.932371 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.932350 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal" event={"ID":"a52f7c3fa8fcd9dbf280379edf7b0272","Type":"ContainerStarted","Data":"012c99aa6f2ed69d2fd6ee0b90bde27aac7fdb1d14ce4f9be269c7ce88a261c1"} Jul 30 02:26:14.934435 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.934406 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" event={"ID":"f62a99b1-985d-4f2a-b516-05e2aa1b6bee","Type":"ContainerStarted","Data":"e4508ac8ee21a24bb546dfcadac9591266a7af28dcf995e9c5bc5aae8b7589b7"} Jul 30 02:26:14.934523 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.934476 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" event={"ID":"f62a99b1-985d-4f2a-b516-05e2aa1b6bee","Type":"ContainerStarted","Data":"942343f9af199f2eb1087eaa297fe04286654dc42bd16f8a97475512c2de6610"} Jul 30 02:26:14.934523 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.934521 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" event={"ID":"f62a99b1-985d-4f2a-b516-05e2aa1b6bee","Type":"ContainerStarted","Data":"d3f5ee24d79d80de66f2589370ecac0be465718f93604cdd31d6714a1da3f34b"} Jul 30 02:26:14.935658 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.935640 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tsrrd" event={"ID":"735cd8d6-2250-4b53-abef-118129c12350","Type":"ContainerStarted","Data":"3587e4328a0f036a0e47023bdee3f18d45d21af861049ab05f7394626f6757fa"} Jul 30 02:26:14.967518 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.967473 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-84dj8" podStartSLOduration=2.758164937 podStartE2EDuration="21.967460912s" totalImagesPullingTime="19.209295975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 02:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:26:14.967051067 +0000 UTC m=+22.645855063" watchObservedRunningTime="2026-07-30 02:26:14.967460912 +0000 UTC m=+22.646264904" Jul 30 02:26:14.991019 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:14.990983 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gmdnj" podStartSLOduration=2.76614389 podStartE2EDuration="21.990969367s" totalImagesPullingTime="19.224825477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 02:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:26:14.990380025 +0000 UTC m=+22.669184016" watchObservedRunningTime="2026-07-30 02:26:14.990969367 +0000 UTC m=+22.669773362" Jul 30 02:26:15.006139 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.005910 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8zfm4" podStartSLOduration=2.8110025629999997 podStartE2EDuration="22.005894599s" totalImagesPullingTime="19.194892036s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 02:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:26:15.005565036 +0000 UTC m=+22.684369034" watchObservedRunningTime="2026-07-30 02:26:15.005894599 +0000 UTC m=+22.684698595" Jul 30 02:26:15.036760 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.036715 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-5.ec2.internal" podStartSLOduration=21.03669943 podStartE2EDuration="21.03669943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 02:25:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:26:15.018711831 +0000 UTC m=+22.697515827" watchObservedRunningTime="2026-07-30 02:26:15.03669943 +0000 UTC m=+22.715503427" Jul 30 02:26:15.037167 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.037140 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tsrrd" podStartSLOduration=2.801995108 podStartE2EDuration="22.037129752s" totalImagesPullingTime="19.235134644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 02:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:26:15.036570744 +0000 UTC m=+22.715374741" watchObservedRunningTime="2026-07-30 02:26:15.037129752 +0000 UTC m=+22.715933747" Jul 30 02:26:15.065739 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.065701 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9l7r4" podStartSLOduration=7.043377046 podStartE2EDuration="22.065687974s" totalImagesPullingTime="15.022310928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:26:15.065431689 +0000 UTC m=+22.744235686" watchObservedRunningTime="2026-07-30 02:26:15.065687974 +0000 UTC m=+22.744491969" Jul 30 02:26:15.847240 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.847180 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:15.847240 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.847217 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:15.847473 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:15.847287 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:15.847473 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:15.847376 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:15.938342 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.938318 2566 generic.go:356] "Generic (PLEG): container finished" podID="71aaec03-629a-427a-9207-6f58e2540c54" containerID="a234f1b1739126264c07de54f2896ae0835d7612f2bfe29316432e4e20de650b" exitCode=0 Jul 30 02:26:15.938768 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.938370 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fpm66" event={"ID":"71aaec03-629a-427a-9207-6f58e2540c54","Type":"ContainerDied","Data":"a234f1b1739126264c07de54f2896ae0835d7612f2bfe29316432e4e20de650b"} Jul 30 02:26:15.941528 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.941510 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" event={"ID":"f62a99b1-985d-4f2a-b516-05e2aa1b6bee","Type":"ContainerStarted","Data":"20007c52c2d233fb4878d0713d8a546abbfb65e1105d35d96e60ab728814e980"} Jul 30 02:26:15.941629 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.941571 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" event={"ID":"f62a99b1-985d-4f2a-b516-05e2aa1b6bee","Type":"ContainerStarted","Data":"59e55c22f59ca9c3b125817bface037cca3834afe513c98e66082383bcf4ea26"} Jul 30 02:26:15.941629 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.941616 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" event={"ID":"f62a99b1-985d-4f2a-b516-05e2aa1b6bee","Type":"ContainerStarted","Data":"cc5decd5ba7faced74d07b44993f2746c71d50b134114976d1cf3e4c85899de7"} Jul 30 02:26:15.943131 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.943110 2566 generic.go:356] "Generic (PLEG): container finished" podID="3409f2fa339f40a2f8e8edf5c4c6e45d" containerID="5e9f9c205262828890a0a426bf98d0b22deffced67985ee9812488086b060253" exitCode=0 Jul 30 02:26:15.943231 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.943163 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" event={"ID":"3409f2fa339f40a2f8e8edf5c4c6e45d","Type":"ContainerDied","Data":"5e9f9c205262828890a0a426bf98d0b22deffced67985ee9812488086b060253"} Jul 30 02:26:15.943231 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.943210 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" event={"ID":"3409f2fa339f40a2f8e8edf5c4c6e45d","Type":"ContainerStarted","Data":"ad88b58f02fc0ac853c4dce2a2ded96ad71959734aa36eee10db4b8617b6a70d"} Jul 30 02:26:15.947274 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.947254 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j4rqd" event={"ID":"95cdb02e-32f7-442b-91da-05b7e9755270","Type":"ContainerStarted","Data":"cb934943a4216ac49cc3f5300a3dab2506d517be59e1f39ea039962f564985a0"} Jul 30 02:26:15.969226 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.969190 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j4rqd" podStartSLOduration=3.795341805 podStartE2EDuration="22.969178897s" totalImagesPullingTime="19.173837092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 02:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:26:15.968941475 +0000 UTC m=+23.647745470" watchObservedRunningTime="2026-07-30 02:26:15.969178897 +0000 UTC m=+23.647982891" Jul 30 02:26:15.982128 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:15.982092 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-5.ec2.internal" podStartSLOduration=21.982081524 podStartE2EDuration="21.982081524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 02:25:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:26:15.981749657 +0000 UTC m=+23.660553649" watchObservedRunningTime="2026-07-30 02:26:15.982081524 +0000 UTC m=+23.660885518" Jul 30 02:26:16.181238 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:16.181218 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 30 02:26:16.365863 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:16.365835 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9l7r4" Jul 30 02:26:16.366341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:16.366323 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9l7r4" Jul 30 02:26:16.790404 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:16.790279 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T02:26:16.181235023Z","UUID":"09ae54e0-5ed9-4e10-8437-0ec297ba62b3","Handler":null,"Name":"","Endpoint":""} Jul 30 02:26:16.792064 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:16.791977 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 30 02:26:16.792064 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:16.792021 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 30 02:26:16.847862 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:16.847236 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:16.847862 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:16.847558 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:16.951731 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:16.951698 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" event={"ID":"3b6d7d15-149c-4ae8-b6d8-741839934430","Type":"ContainerStarted","Data":"926f7be709ca3068a3c9b421ccd11c4994c595fda99277fb0c4107f75851ffac"} Jul 30 02:26:16.952265 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:16.952231 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9l7r4" Jul 30 02:26:16.952757 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:16.952737 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9l7r4" Jul 30 02:26:17.847061 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:17.846982 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:17.847061 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:17.847024 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:17.847319 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:17.847105 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:17.847319 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:17.847237 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:17.957291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:17.957257 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" event={"ID":"f62a99b1-985d-4f2a-b516-05e2aa1b6bee","Type":"ContainerStarted","Data":"6cb6d3efabb38169bd8b47c452fe198209ddb957822a6a735dec14551fc44875"} Jul 30 02:26:17.961479 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:17.961448 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" event={"ID":"3b6d7d15-149c-4ae8-b6d8-741839934430","Type":"ContainerStarted","Data":"9c82a99c4edfefdd8a4bc44d5189fc7130afc3b197b1e0003c63e2fa8894dc6a"} Jul 30 02:26:17.978997 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:17.978955 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5xcm" podStartSLOduration=3.255784973 podStartE2EDuration="24.978941865s" totalImagesPullingTime="21.723156892s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 02:25:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:26:17.97863012 +0000 UTC m=+25.657434116" watchObservedRunningTime="2026-07-30 02:26:17.978941865 +0000 UTC m=+25.657745909" Jul 30 02:26:18.846863 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:18.846829 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:18.847038 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:18.846968 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:19.846698 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:19.846653 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:19.847130 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:19.846702 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:19.847130 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:19.846788 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:19.847130 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:19.846921 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:20.847048 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:20.846892 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:20.847703 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:20.847142 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:20.968070 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:20.968017 2566 generic.go:356] "Generic (PLEG): container finished" podID="71aaec03-629a-427a-9207-6f58e2540c54" containerID="a10f09a27864b17625a6bc44bf063d826bc268090f9c0b979ccf9c6869f5c35c" exitCode=0 Jul 30 02:26:20.968164 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:20.968092 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fpm66" event={"ID":"71aaec03-629a-427a-9207-6f58e2540c54","Type":"ContainerDied","Data":"a10f09a27864b17625a6bc44bf063d826bc268090f9c0b979ccf9c6869f5c35c"} Jul 30 02:26:20.971825 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:20.971804 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" event={"ID":"f62a99b1-985d-4f2a-b516-05e2aa1b6bee","Type":"ContainerStarted","Data":"aba02ccce4ec1d800557ecae568346d4f7d09c5ce9153943dc4f3faadacac562"} Jul 30 02:26:20.972045 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:20.972033 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:26:20.986989 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:20.986972 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:26:21.016061 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:21.016013 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" podStartSLOduration=8.437590137 podStartE2EDuration="28.015999141s" totalImagesPullingTime="19.578409004s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 02:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:26:21.012900393 +0000 UTC m=+28.691704370" watchObservedRunningTime="2026-07-30 02:26:21.015999141 +0000 UTC m=+28.694803130" Jul 30 02:26:21.846681 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:21.846637 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:21.846941 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:21.846772 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:21.846941 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:21.846821 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:21.846941 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:21.846919 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:21.973930 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:21.973908 2566 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 30 02:26:21.974422 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:21.974401 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:26:21.992264 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:21.992241 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:26:22.295634 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:22.295453 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:26:22.366141 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:22.366114 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dp2xm"] Jul 30 02:26:22.366301 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:22.366222 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:22.366357 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:22.366317 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:22.366907 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:22.366884 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z6stl"] Jul 30 02:26:22.367001 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:22.366960 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:22.367080 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:22.367055 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:22.369081 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:22.369064 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rx95s"] Jul 30 02:26:22.369176 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:22.369148 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:22.369266 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:22.369227 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:22.977002 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:22.976973 2566 generic.go:356] "Generic (PLEG): container finished" podID="71aaec03-629a-427a-9207-6f58e2540c54" containerID="d75a7a86fc9497169da062a45e093ed652b73ade2e26a6a05990687f17d41781" exitCode=0 Jul 30 02:26:22.977417 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:22.977057 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fpm66" event={"ID":"71aaec03-629a-427a-9207-6f58e2540c54","Type":"ContainerDied","Data":"d75a7a86fc9497169da062a45e093ed652b73ade2e26a6a05990687f17d41781"} Jul 30 02:26:23.847472 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:23.847412 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:23.847599 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:23.847497 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:23.847599 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:23.847503 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:23.847599 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:23.847524 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:23.847736 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:23.847606 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:23.847736 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:23.847697 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:24.984790 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:24.984760 2566 generic.go:356] "Generic (PLEG): container finished" podID="71aaec03-629a-427a-9207-6f58e2540c54" containerID="1bbb7308bf9706cab4a59542bde470c3af9daba5448e49f145ece2c0f36e1f0c" exitCode=0 Jul 30 02:26:24.985177 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:24.984809 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fpm66" event={"ID":"71aaec03-629a-427a-9207-6f58e2540c54","Type":"ContainerDied","Data":"1bbb7308bf9706cab4a59542bde470c3af9daba5448e49f145ece2c0f36e1f0c"} Jul 30 02:26:25.846685 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:25.846616 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:25.846860 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:25.846736 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:25.846860 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:25.846749 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z6stl" podUID="32d47ce4-b22b-4416-b81d-3d12743dbdad" Jul 30 02:26:25.846860 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:25.846771 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:25.847012 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:25.846882 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:26:25.847012 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:25.846967 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2xm" podUID="1aca484e-f11d-45ae-acbb-e227a511b00c" Jul 30 02:26:26.488640 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:26.488600 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:26.489236 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:26.488712 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:26.489236 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:26.488792 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:26:26.489236 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:26.488808 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 02:26:26.489236 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:26.488869 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs podName:db23d664-ec37-4206-b1f3-ac52ecec65dc nodeName:}" failed. No retries permitted until 2026-07-30 02:26:58.488847633 +0000 UTC m=+66.167651612 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs") pod "network-metrics-daemon-rx95s" (UID: "db23d664-ec37-4206-b1f3-ac52ecec65dc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 02:26:26.489236 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:26.488889 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret podName:1aca484e-f11d-45ae-acbb-e227a511b00c nodeName:}" failed. No retries permitted until 2026-07-30 02:26:58.488879342 +0000 UTC m=+66.167683321 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret") pod "global-pull-secret-syncer-dp2xm" (UID: "1aca484e-f11d-45ae-acbb-e227a511b00c") : object "kube-system"/"original-pull-secret" not registered Jul 30 02:26:26.690154 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:26.690116 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfg5f\" (UniqueName: \"kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f\") pod \"network-check-target-z6stl\" (UID: \"32d47ce4-b22b-4416-b81d-3d12743dbdad\") " pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:26.690339 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:26.690285 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 02:26:26.690339 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:26.690307 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 02:26:26.690339 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:26.690320 2566 projected.go:196] Error preparing data for projected volume kube-api-access-lfg5f for pod openshift-network-diagnostics/network-check-target-z6stl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:26:26.690500 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:26.690369 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f podName:32d47ce4-b22b-4416-b81d-3d12743dbdad nodeName:}" failed. No retries permitted until 2026-07-30 02:26:58.690356499 +0000 UTC m=+66.369160472 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lfg5f" (UniqueName: "kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f") pod "network-check-target-z6stl" (UID: "32d47ce4-b22b-4416-b81d-3d12743dbdad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 02:26:27.139887 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.139860 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-5.ec2.internal" event="NodeReady" Jul 30 02:26:27.140071 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.140001 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 30 02:26:27.179037 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.179001 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-rr58w"] Jul 30 02:26:27.204374 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.204350 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c7765b4fc-p4gvx"] Jul 30 02:26:27.204534 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.204480 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:26:27.207163 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.207136 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 30 02:26:27.207330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.207137 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 30 02:26:27.207330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.207148 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-62dmk\"" Jul 30 02:26:27.218781 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.218759 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-rr58w"] Jul 30 02:26:27.218914 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.218796 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c7765b4fc-p4gvx"] Jul 30 02:26:27.218914 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.218818 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7pj54"] Jul 30 02:26:27.219390 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.219041 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.222260 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.222242 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 30 02:26:27.222732 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.222713 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 30 02:26:27.222998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.222926 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 30 02:26:27.223238 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.223222 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6r6b9\"" Jul 30 02:26:27.236868 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.236440 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 30 02:26:27.237154 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.237136 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7rjk5"] Jul 30 02:26:27.237270 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.237249 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7pj54" Jul 30 02:26:27.237270 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.237230 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:27.239780 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.239719 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tg8zh\"" Jul 30 02:26:27.239780 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.239764 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 30 02:26:27.239955 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.239937 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 30 02:26:27.240053 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.240009 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 30 02:26:27.259397 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.259342 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7pj54"] Jul 30 02:26:27.259397 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.259379 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7rjk5"] Jul 30 02:26:27.259562 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.259426 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:27.263223 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.263201 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 30 02:26:27.263332 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.263211 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 30 02:26:27.263332 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.263284 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5sqnz\"" Jul 30 02:26:27.294264 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.294240 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:26:27.294413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.294310 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1b389afb-4064-40ac-a473-89da51b1f974-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:26:27.349720 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.349657 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:27.395627 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.395594 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq2xw\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-kube-api-access-bq2xw\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.395627 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.395631 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8swx6\" (UniqueName: \"kubernetes.io/projected/d67b207e-c987-490a-9051-3006d810dee3-kube-api-access-8swx6\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:27.395894 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.395707 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d67b207e-c987-490a-9051-3006d810dee3-tmp-dir\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:27.395894 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.395735 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:26:27.395894 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.395777 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ca64202a-9a1b-42d4-b980-cabdcddc4976-ca-trust-extracted\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.395894 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.395808 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmkpw\" (UniqueName: \"kubernetes.io/projected/b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0-kube-api-access-gmkpw\") pod \"ingress-canary-7pj54\" (UID: \"b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0\") " pod="openshift-ingress-canary/ingress-canary-7pj54" Jul 30 02:26:27.395894 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:27.395826 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 02:26:27.395894 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:27.395870 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert podName:1b389afb-4064-40ac-a473-89da51b1f974 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:27.895856284 +0000 UTC m=+35.574660257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rr58w" (UID: "1b389afb-4064-40ac-a473-89da51b1f974") : secret "networking-console-plugin-cert" not found Jul 30 02:26:27.396290 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.395935 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-certificates\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.396290 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.395976 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ca64202a-9a1b-42d4-b980-cabdcddc4976-installation-pull-secrets\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.396290 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.396026 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-bound-sa-token\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.396290 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.396051 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d67b207e-c987-490a-9051-3006d810dee3-config-volume\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:27.396290 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.396102 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ca64202a-9a1b-42d4-b980-cabdcddc4976-image-registry-private-configuration\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.396290 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.396137 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ca64202a-9a1b-42d4-b980-cabdcddc4976-trusted-ca\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.396290 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.396163 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:27.396290 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.396196 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1b389afb-4064-40ac-a473-89da51b1f974-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:26:27.396290 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.396229 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.396290 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.396258 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0-cert\") pod \"ingress-canary-7pj54\" (UID: \"b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0\") " pod="openshift-ingress-canary/ingress-canary-7pj54" Jul 30 02:26:27.396990 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.396970 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1b389afb-4064-40ac-a473-89da51b1f974-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:26:27.497424 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.497381 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ca64202a-9a1b-42d4-b980-cabdcddc4976-installation-pull-secrets\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.497854 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.497443 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-bound-sa-token\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.497854 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.497468 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d67b207e-c987-490a-9051-3006d810dee3-config-volume\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:27.497854 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.497497 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ca64202a-9a1b-42d4-b980-cabdcddc4976-image-registry-private-configuration\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.497854 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.497518 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ca64202a-9a1b-42d4-b980-cabdcddc4976-trusted-ca\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.497854 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.497700 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:27.497854 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.497759 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.497854 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.497785 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0-cert\") pod \"ingress-canary-7pj54\" (UID: \"b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0\") " pod="openshift-ingress-canary/ingress-canary-7pj54" Jul 30 02:26:27.497854 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.497833 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq2xw\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-kube-api-access-bq2xw\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.498250 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.497868 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8swx6\" (UniqueName: \"kubernetes.io/projected/d67b207e-c987-490a-9051-3006d810dee3-kube-api-access-8swx6\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:27.498250 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.497915 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d67b207e-c987-490a-9051-3006d810dee3-tmp-dir\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:27.498250 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:27.497933 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 02:26:27.498250 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:27.497952 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c7765b4fc-p4gvx: secret "image-registry-tls" not found Jul 30 02:26:27.498250 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.497959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ca64202a-9a1b-42d4-b980-cabdcddc4976-ca-trust-extracted\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.498250 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.497983 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmkpw\" (UniqueName: \"kubernetes.io/projected/b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0-kube-api-access-gmkpw\") pod \"ingress-canary-7pj54\" (UID: \"b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0\") " pod="openshift-ingress-canary/ingress-canary-7pj54" Jul 30 02:26:27.498250 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:27.498009 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls podName:ca64202a-9a1b-42d4-b980-cabdcddc4976 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:27.997988996 +0000 UTC m=+35.676792987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls") pod "image-registry-6c7765b4fc-p4gvx" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976") : secret "image-registry-tls" not found Jul 30 02:26:27.498250 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.498077 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-certificates\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.498250 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.498209 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d67b207e-c987-490a-9051-3006d810dee3-config-volume\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:27.498719 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:27.498338 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 02:26:27.498719 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:27.498399 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls podName:d67b207e-c987-490a-9051-3006d810dee3 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:27.998381329 +0000 UTC m=+35.677185316 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls") pod "dns-default-7rjk5" (UID: "d67b207e-c987-490a-9051-3006d810dee3") : secret "dns-default-metrics-tls" not found Jul 30 02:26:27.498719 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.498593 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-certificates\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.498719 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.498599 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ca64202a-9a1b-42d4-b980-cabdcddc4976-trusted-ca\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.498929 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.498794 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d67b207e-c987-490a-9051-3006d810dee3-tmp-dir\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:27.498929 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.498903 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ca64202a-9a1b-42d4-b980-cabdcddc4976-ca-trust-extracted\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.502536 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.502399 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ca64202a-9a1b-42d4-b980-cabdcddc4976-image-registry-private-configuration\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.502645 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.502402 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ca64202a-9a1b-42d4-b980-cabdcddc4976-installation-pull-secrets\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.502645 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.502444 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0-cert\") pod \"ingress-canary-7pj54\" (UID: \"b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0\") " pod="openshift-ingress-canary/ingress-canary-7pj54" Jul 30 02:26:27.505691 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.505651 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-bound-sa-token\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.507361 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.507339 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8swx6\" (UniqueName: \"kubernetes.io/projected/d67b207e-c987-490a-9051-3006d810dee3-kube-api-access-8swx6\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:27.507535 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.507520 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq2xw\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-kube-api-access-bq2xw\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:27.508051 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.508032 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmkpw\" (UniqueName: \"kubernetes.io/projected/b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0-kube-api-access-gmkpw\") pod \"ingress-canary-7pj54\" (UID: \"b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0\") " pod="openshift-ingress-canary/ingress-canary-7pj54" Jul 30 02:26:27.547864 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.547804 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7pj54" Jul 30 02:26:27.721814 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.721775 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7pj54"] Jul 30 02:26:27.846683 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.846589 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:27.846683 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.846590 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:27.846911 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.846589 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:27.849562 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.849520 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 02:26:27.849707 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.849605 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vvjjn\"" Jul 30 02:26:27.849707 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.849645 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 30 02:26:27.849707 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.849677 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 02:26:27.849918 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.849763 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-67lr8\"" Jul 30 02:26:27.850004 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.849985 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 30 02:26:27.901398 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:27.901365 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:26:27.901542 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:27.901521 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 02:26:27.901597 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:27.901586 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert podName:1b389afb-4064-40ac-a473-89da51b1f974 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:28.901566028 +0000 UTC m=+36.580370001 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rr58w" (UID: "1b389afb-4064-40ac-a473-89da51b1f974") : secret "networking-console-plugin-cert" not found Jul 30 02:26:28.002467 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:28.002200 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:28.002467 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:28.002253 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:28.002467 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:28.002394 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 02:26:28.002467 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:28.002405 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 02:26:28.002467 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:28.002408 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c7765b4fc-p4gvx: secret "image-registry-tls" not found Jul 30 02:26:28.002467 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:28.002464 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls podName:ca64202a-9a1b-42d4-b980-cabdcddc4976 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:29.002446854 +0000 UTC m=+36.681250845 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls") pod "image-registry-6c7765b4fc-p4gvx" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976") : secret "image-registry-tls" not found Jul 30 02:26:28.002887 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:28.002485 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls podName:d67b207e-c987-490a-9051-3006d810dee3 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:29.002473571 +0000 UTC m=+36.681277543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls") pod "dns-default-7rjk5" (UID: "d67b207e-c987-490a-9051-3006d810dee3") : secret "dns-default-metrics-tls" not found Jul 30 02:26:28.029389 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:28.029347 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:28.129218 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:28.129133 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:28.345277 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:28.345240 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:28.402249 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:28.402172 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:28.911377 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:28.911343 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:26:28.912116 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:28.911513 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 02:26:28.912116 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:28.911581 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert podName:1b389afb-4064-40ac-a473-89da51b1f974 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:30.911563821 +0000 UTC m=+38.590367808 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rr58w" (UID: "1b389afb-4064-40ac-a473-89da51b1f974") : secret "networking-console-plugin-cert" not found Jul 30 02:26:29.012358 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:29.012326 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:29.012520 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:29.012379 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:29.012520 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:29.012494 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 02:26:29.012650 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:29.012567 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls podName:d67b207e-c987-490a-9051-3006d810dee3 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:31.012546759 +0000 UTC m=+38.691350760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls") pod "dns-default-7rjk5" (UID: "d67b207e-c987-490a-9051-3006d810dee3") : secret "dns-default-metrics-tls" not found Jul 30 02:26:29.012650 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:29.012577 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 02:26:29.012650 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:29.012597 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c7765b4fc-p4gvx: secret "image-registry-tls" not found Jul 30 02:26:29.012807 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:29.012656 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls podName:ca64202a-9a1b-42d4-b980-cabdcddc4976 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:31.012639882 +0000 UTC m=+38.691443855 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls") pod "image-registry-6c7765b4fc-p4gvx" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976") : secret "image-registry-tls" not found Jul 30 02:26:29.155361 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:29.155308 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:29.250361 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:26:29.250334 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d2ebcc_f9c2_414e_8bb7_66ca254e5fe0.slice/crio-3c997b6bfef09fa500342180b6bcd0dbe5073ae2a2bf41a07fce57d781931100 WatchSource:0}: Error finding container 3c997b6bfef09fa500342180b6bcd0dbe5073ae2a2bf41a07fce57d781931100: Status 404 returned error can't find the container with id 3c997b6bfef09fa500342180b6bcd0dbe5073ae2a2bf41a07fce57d781931100 Jul 30 02:26:29.553222 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:29.553133 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:29.606310 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:29.606284 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:29.999752 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:29.999713 2566 generic.go:356] "Generic (PLEG): container finished" podID="71aaec03-629a-427a-9207-6f58e2540c54" containerID="66f12a33a195147866a27993af2365798134f9fde9c58501f848bc0e4f64d761" exitCode=0 Jul 30 02:26:30.000385 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:29.999799 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fpm66" event={"ID":"71aaec03-629a-427a-9207-6f58e2540c54","Type":"ContainerDied","Data":"66f12a33a195147866a27993af2365798134f9fde9c58501f848bc0e4f64d761"} Jul 30 02:26:30.000859 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:30.000841 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7pj54" event={"ID":"b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0","Type":"ContainerStarted","Data":"3c997b6bfef09fa500342180b6bcd0dbe5073ae2a2bf41a07fce57d781931100"} Jul 30 02:26:30.931619 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:30.931422 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:26:30.931814 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:30.931569 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 02:26:30.931814 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:30.931747 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert podName:1b389afb-4064-40ac-a473-89da51b1f974 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:34.931730981 +0000 UTC m=+42.610534957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rr58w" (UID: "1b389afb-4064-40ac-a473-89da51b1f974") : secret "networking-console-plugin-cert" not found Jul 30 02:26:31.004621 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:31.004588 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:31.005930 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:31.005902 2566 generic.go:356] "Generic (PLEG): container finished" podID="71aaec03-629a-427a-9207-6f58e2540c54" containerID="df02b36a33794e5ab2df18ac981fa576342bcddc9c0b06e09969c685087d6e95" exitCode=0 Jul 30 02:26:31.006025 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:31.005964 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fpm66" event={"ID":"71aaec03-629a-427a-9207-6f58e2540c54","Type":"ContainerDied","Data":"df02b36a33794e5ab2df18ac981fa576342bcddc9c0b06e09969c685087d6e95"} Jul 30 02:26:31.032555 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:31.032527 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:31.032642 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:31.032579 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:31.032716 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:31.032657 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 02:26:31.032716 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:31.032693 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 02:26:31.032716 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:31.032705 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c7765b4fc-p4gvx: secret "image-registry-tls" not found Jul 30 02:26:31.032814 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:31.032732 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls podName:d67b207e-c987-490a-9051-3006d810dee3 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:35.032713043 +0000 UTC m=+42.711517021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls") pod "dns-default-7rjk5" (UID: "d67b207e-c987-490a-9051-3006d810dee3") : secret "dns-default-metrics-tls" not found Jul 30 02:26:31.032945 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:31.032928 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls podName:ca64202a-9a1b-42d4-b980-cabdcddc4976 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:35.032908174 +0000 UTC m=+42.711712160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls") pod "image-registry-6c7765b4fc-p4gvx" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976") : secret "image-registry-tls" not found Jul 30 02:26:31.871551 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:31.871509 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:31.972046 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:31.972003 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:32.011382 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:32.011187 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fpm66" event={"ID":"71aaec03-629a-427a-9207-6f58e2540c54","Type":"ContainerStarted","Data":"911bc752f63c193ba7e87aef5ce3cc68bd9f550c6754639450913b6ef9634625"} Jul 30 02:26:32.034899 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:32.034160 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fpm66" podStartSLOduration=16.605672511 podStartE2EDuration="39.034135073s" totalImagesPullingTime="22.428462562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 02:25:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:26:32.032373209 +0000 UTC m=+39.711177207" watchObservedRunningTime="2026-07-30 02:26:32.034135073 +0000 UTC m=+39.712939071" Jul 30 02:26:32.172261 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:32.172181 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:32.273360 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:32.273327 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:33.014398 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:33.014363 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7pj54" event={"ID":"b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0","Type":"ContainerStarted","Data":"520648a3d520d4e3122517d4052c0ebe7b3757f19a000902ca5810f43a0bc108"} Jul 30 02:26:33.030086 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:33.030021 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7pj54" podStartSLOduration=3.269533318 podStartE2EDuration="6.030003737s" totalImagesPullingTime="2.760470419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:26:33.029330859 +0000 UTC m=+40.708134854" watchObservedRunningTime="2026-07-30 02:26:33.030003737 +0000 UTC m=+40.708807735" Jul 30 02:26:34.958913 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:34.958875 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:26:34.959285 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:34.959033 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 02:26:34.959285 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:34.959100 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert podName:1b389afb-4064-40ac-a473-89da51b1f974 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:42.959083435 +0000 UTC m=+50.637887413 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rr58w" (UID: "1b389afb-4064-40ac-a473-89da51b1f974") : secret "networking-console-plugin-cert" not found Jul 30 02:26:35.059645 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:35.059606 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:35.059645 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:35.059647 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:35.059834 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:35.059758 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 02:26:35.059834 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:35.059820 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls podName:d67b207e-c987-490a-9051-3006d810dee3 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:43.059805714 +0000 UTC m=+50.738609688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls") pod "dns-default-7rjk5" (UID: "d67b207e-c987-490a-9051-3006d810dee3") : secret "dns-default-metrics-tls" not found Jul 30 02:26:35.059919 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:35.059762 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 02:26:35.059919 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:35.059854 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c7765b4fc-p4gvx: secret "image-registry-tls" not found Jul 30 02:26:35.059919 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:35.059899 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls podName:ca64202a-9a1b-42d4-b980-cabdcddc4976 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:43.059888158 +0000 UTC m=+50.738692132 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls") pod "image-registry-6c7765b4fc-p4gvx" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976") : secret "image-registry-tls" not found Jul 30 02:26:35.177304 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:35.177275 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:35.234166 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:35.234091 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:40.169264 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:40.169228 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:40.227808 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:40.227781 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:40.880654 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:40.880622 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:40.939672 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:40.939630 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:43.019487 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:43.019457 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:26:43.019875 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:43.019565 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 02:26:43.019875 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:43.019623 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert podName:1b389afb-4064-40ac-a473-89da51b1f974 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:59.019610636 +0000 UTC m=+66.698414608 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rr58w" (UID: "1b389afb-4064-40ac-a473-89da51b1f974") : secret "networking-console-plugin-cert" not found Jul 30 02:26:43.050154 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:43.050122 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:43.106639 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:43.106612 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:43.119770 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:43.119748 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:43.119849 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:43.119778 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:43.119896 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:43.119881 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 02:26:43.119896 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:43.119885 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 02:26:43.119961 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:43.119933 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls podName:d67b207e-c987-490a-9051-3006d810dee3 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:59.119920143 +0000 UTC m=+66.798724119 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls") pod "dns-default-7rjk5" (UID: "d67b207e-c987-490a-9051-3006d810dee3") : secret "dns-default-metrics-tls" not found Jul 30 02:26:43.120007 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:43.119892 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c7765b4fc-p4gvx: secret "image-registry-tls" not found Jul 30 02:26:43.120040 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:43.120010 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls podName:ca64202a-9a1b-42d4-b980-cabdcddc4976 nodeName:}" failed. No retries permitted until 2026-07-30 02:26:59.119998124 +0000 UTC m=+66.798802097 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls") pod "image-registry-6c7765b4fc-p4gvx" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976") : secret "image-registry-tls" not found Jul 30 02:26:43.479103 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:43.479018 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:43.579437 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:43.579405 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:43.837531 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:43.837506 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:43.889394 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:43.889369 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:44.115612 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:44.115546 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:44.216518 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:44.216494 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:54.000983 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:54.000956 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkwg7" Jul 30 02:26:54.149479 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:54.149453 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:54.205449 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:54.205419 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:26:58.523439 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.523387 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:58.523439 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.523440 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:26:58.526353 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.526332 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 02:26:58.526464 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.526381 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 02:26:58.534259 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:58.534237 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 30 02:26:58.534347 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:58.534315 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs podName:db23d664-ec37-4206-b1f3-ac52ecec65dc nodeName:}" failed. No retries permitted until 2026-07-30 02:28:02.534294963 +0000 UTC m=+130.213098949 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs") pod "network-metrics-daemon-rx95s" (UID: "db23d664-ec37-4206-b1f3-ac52ecec65dc") : secret "metrics-daemon-secret" not found Jul 30 02:26:58.537279 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.537261 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1aca484e-f11d-45ae-acbb-e227a511b00c-original-pull-secret\") pod \"global-pull-secret-syncer-dp2xm\" (UID: \"1aca484e-f11d-45ae-acbb-e227a511b00c\") " pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:58.725714 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.725681 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfg5f\" (UniqueName: \"kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f\") pod \"network-check-target-z6stl\" (UID: \"32d47ce4-b22b-4416-b81d-3d12743dbdad\") " pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:58.728684 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.728650 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 30 02:26:58.738007 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.737984 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 30 02:26:58.748977 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.748955 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfg5f\" (UniqueName: \"kubernetes.io/projected/32d47ce4-b22b-4416-b81d-3d12743dbdad-kube-api-access-lfg5f\") pod \"network-check-target-z6stl\" (UID: \"32d47ce4-b22b-4416-b81d-3d12743dbdad\") " pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:58.759852 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.759834 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2xm" Jul 30 02:26:58.767839 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.767819 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-67lr8\"" Jul 30 02:26:58.776463 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.776411 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:26:58.903760 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.902050 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dp2xm"] Jul 30 02:26:58.907319 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:26:58.907294 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aca484e_f11d_45ae_acbb_e227a511b00c.slice/crio-9f430850d79b08288a9bbb60f5c15c17a0166bdc526ba08201283f4b8af4e5fe WatchSource:0}: Error finding container 9f430850d79b08288a9bbb60f5c15c17a0166bdc526ba08201283f4b8af4e5fe: Status 404 returned error can't find the container with id 9f430850d79b08288a9bbb60f5c15c17a0166bdc526ba08201283f4b8af4e5fe Jul 30 02:26:58.921162 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:58.921138 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z6stl"] Jul 30 02:26:58.923766 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:26:58.923738 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32d47ce4_b22b_4416_b81d_3d12743dbdad.slice/crio-df69441656636dec8f393c700238d2dce01392b78935098ea9986bc3d90fc8a7 WatchSource:0}: Error finding container df69441656636dec8f393c700238d2dce01392b78935098ea9986bc3d90fc8a7: Status 404 returned error can't find the container with id df69441656636dec8f393c700238d2dce01392b78935098ea9986bc3d90fc8a7 Jul 30 02:26:59.028101 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:59.028026 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:26:59.028228 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:59.028153 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 02:26:59.028288 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:59.028229 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert podName:1b389afb-4064-40ac-a473-89da51b1f974 nodeName:}" failed. No retries permitted until 2026-07-30 02:27:31.028210249 +0000 UTC m=+98.707014236 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rr58w" (UID: "1b389afb-4064-40ac-a473-89da51b1f974") : secret "networking-console-plugin-cert" not found Jul 30 02:26:59.073113 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:59.073080 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z6stl" event={"ID":"32d47ce4-b22b-4416-b81d-3d12743dbdad","Type":"ContainerStarted","Data":"df69441656636dec8f393c700238d2dce01392b78935098ea9986bc3d90fc8a7"} Jul 30 02:26:59.074225 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:59.074202 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dp2xm" event={"ID":"1aca484e-f11d-45ae-acbb-e227a511b00c","Type":"ContainerStarted","Data":"9f430850d79b08288a9bbb60f5c15c17a0166bdc526ba08201283f4b8af4e5fe"} Jul 30 02:26:59.128615 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:59.128589 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:26:59.128701 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:26:59.128676 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:26:59.128761 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:59.128742 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 02:26:59.128798 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:59.128762 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c7765b4fc-p4gvx: secret "image-registry-tls" not found Jul 30 02:26:59.128798 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:59.128763 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 02:26:59.128872 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:59.128809 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls podName:ca64202a-9a1b-42d4-b980-cabdcddc4976 nodeName:}" failed. No retries permitted until 2026-07-30 02:27:31.128794962 +0000 UTC m=+98.807598938 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls") pod "image-registry-6c7765b4fc-p4gvx" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976") : secret "image-registry-tls" not found Jul 30 02:26:59.128872 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:26:59.128822 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls podName:d67b207e-c987-490a-9051-3006d810dee3 nodeName:}" failed. No retries permitted until 2026-07-30 02:27:31.128816234 +0000 UTC m=+98.807620207 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls") pod "dns-default-7rjk5" (UID: "d67b207e-c987-490a-9051-3006d810dee3") : secret "dns-default-metrics-tls" not found Jul 30 02:27:04.089072 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:04.089026 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z6stl" event={"ID":"32d47ce4-b22b-4416-b81d-3d12743dbdad","Type":"ContainerStarted","Data":"585fcfdc148ceb80767f9ae7d9b18c7c36a2c094f8dae102afecf3f01368d6e0"} Jul 30 02:27:04.089503 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:04.089175 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:27:04.090407 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:04.090384 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dp2xm" event={"ID":"1aca484e-f11d-45ae-acbb-e227a511b00c","Type":"ContainerStarted","Data":"f1b243aada86f7a9d59a43e97eb231de21ddaaf1ab21aca36efa791e489ffba1"} Jul 30 02:27:04.108216 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:04.106029 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z6stl" podStartSLOduration=66.779203095 podStartE2EDuration="1m11.106011191s" totalImagesPullingTime="4.326808096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:27:04.103589525 +0000 UTC m=+71.782393520" watchObservedRunningTime="2026-07-30 02:27:04.106011191 +0000 UTC m=+71.784815187" Jul 30 02:27:04.119985 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:04.119948 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dp2xm" podStartSLOduration=66.773211412 podStartE2EDuration="1m11.119940176s" totalImagesPullingTime="4.346728764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 02:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:27:04.11953902 +0000 UTC m=+71.798343016" watchObservedRunningTime="2026-07-30 02:27:04.119940176 +0000 UTC m=+71.798744170" Jul 30 02:27:06.020837 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:06.020808 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:27:06.086658 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:06.086631 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:27:06.187958 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:06.187925 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:27:06.242391 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:06.242374 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:27:31.043304 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:31.043265 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:27:31.043786 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:27:31.043415 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 02:27:31.043786 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:27:31.043498 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert podName:1b389afb-4064-40ac-a473-89da51b1f974 nodeName:}" failed. No retries permitted until 2026-07-30 02:28:35.043482023 +0000 UTC m=+162.722285999 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rr58w" (UID: "1b389afb-4064-40ac-a473-89da51b1f974") : secret "networking-console-plugin-cert" not found Jul 30 02:27:31.143615 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:31.143582 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:27:31.143735 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:31.143623 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:27:31.143778 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:27:31.143752 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 02:27:31.143778 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:27:31.143754 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 02:27:31.143859 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:27:31.143764 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c7765b4fc-p4gvx: secret "image-registry-tls" not found Jul 30 02:27:31.143903 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:27:31.143851 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls podName:d67b207e-c987-490a-9051-3006d810dee3 nodeName:}" failed. No retries permitted until 2026-07-30 02:28:35.143836046 +0000 UTC m=+162.822640019 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls") pod "dns-default-7rjk5" (UID: "d67b207e-c987-490a-9051-3006d810dee3") : secret "dns-default-metrics-tls" not found Jul 30 02:27:31.143943 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:27:31.143916 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls podName:ca64202a-9a1b-42d4-b980-cabdcddc4976 nodeName:}" failed. No retries permitted until 2026-07-30 02:28:35.143903165 +0000 UTC m=+162.822707137 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls") pod "image-registry-6c7765b4fc-p4gvx" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976") : secret "image-registry-tls" not found Jul 30 02:27:31.222603 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:31.222565 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:27:31.278467 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:31.278437 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:27:35.095436 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:27:35.095406 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z6stl" Jul 30 02:28:02.556945 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:02.556904 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:28:02.557541 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:28:02.557065 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 30 02:28:02.557541 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:28:02.557145 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs podName:db23d664-ec37-4206-b1f3-ac52ecec65dc nodeName:}" failed. No retries permitted until 2026-07-30 02:30:04.557123971 +0000 UTC m=+252.235927963 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs") pod "network-metrics-daemon-rx95s" (UID: "db23d664-ec37-4206-b1f3-ac52ecec65dc") : secret "metrics-daemon-secret" not found Jul 30 02:28:30.217425 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:28:30.217379 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" podUID="1b389afb-4064-40ac-a473-89da51b1f974" Jul 30 02:28:30.237550 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:28:30.237512 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" podUID="ca64202a-9a1b-42d4-b980-cabdcddc4976" Jul 30 02:28:30.268822 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:28:30.268794 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7rjk5" podUID="d67b207e-c987-490a-9051-3006d810dee3" Jul 30 02:28:30.288656 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:30.288631 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:28:30.288773 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:30.288631 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7rjk5" Jul 30 02:28:30.871902 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:28:30.871857 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:28:35.074520 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.074477 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:28:35.074955 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:28:35.074627 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 02:28:35.074955 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:28:35.074742 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert podName:1b389afb-4064-40ac-a473-89da51b1f974 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:37.074724903 +0000 UTC m=+284.753528880 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-rr58w" (UID: "1b389afb-4064-40ac-a473-89da51b1f974") : secret "networking-console-plugin-cert" not found Jul 30 02:28:35.175743 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.175711 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:28:35.175881 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.175767 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:28:35.175881 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:28:35.175844 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 02:28:35.175965 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:28:35.175907 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls podName:d67b207e-c987-490a-9051-3006d810dee3 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:37.175889931 +0000 UTC m=+284.854693903 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls") pod "dns-default-7rjk5" (UID: "d67b207e-c987-490a-9051-3006d810dee3") : secret "dns-default-metrics-tls" not found Jul 30 02:28:35.175965 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:28:35.175912 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 02:28:35.175965 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:28:35.175928 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c7765b4fc-p4gvx: secret "image-registry-tls" not found Jul 30 02:28:35.176075 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:28:35.175977 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls podName:ca64202a-9a1b-42d4-b980-cabdcddc4976 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:37.175961557 +0000 UTC m=+284.854765531 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls") pod "image-registry-6c7765b4fc-p4gvx" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976") : secret "image-registry-tls" not found Jul 30 02:28:35.511266 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.511232 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt"] Jul 30 02:28:35.513102 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.513087 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:28:35.516582 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.516559 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 30 02:28:35.516734 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.516600 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 30 02:28:35.516734 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.516564 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 30 02:28:35.516734 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.516626 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 30 02:28:35.523655 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.523633 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt"] Jul 30 02:28:35.578934 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.578896 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rxcf\" (UniqueName: \"kubernetes.io/projected/a4ca0928-2f8b-4840-aa21-fb8e23d1865d-kube-api-access-7rxcf\") pod \"klusterlet-addon-workmgr-6c45fbd6df-49fzt\" (UID: \"a4ca0928-2f8b-4840-aa21-fb8e23d1865d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:28:35.578934 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.578932 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a4ca0928-2f8b-4840-aa21-fb8e23d1865d-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c45fbd6df-49fzt\" (UID: \"a4ca0928-2f8b-4840-aa21-fb8e23d1865d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:28:35.579122 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.579038 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4ca0928-2f8b-4840-aa21-fb8e23d1865d-tmp\") pod \"klusterlet-addon-workmgr-6c45fbd6df-49fzt\" (UID: \"a4ca0928-2f8b-4840-aa21-fb8e23d1865d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:28:35.679981 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.679934 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rxcf\" (UniqueName: \"kubernetes.io/projected/a4ca0928-2f8b-4840-aa21-fb8e23d1865d-kube-api-access-7rxcf\") pod \"klusterlet-addon-workmgr-6c45fbd6df-49fzt\" (UID: \"a4ca0928-2f8b-4840-aa21-fb8e23d1865d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:28:35.679981 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.679981 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a4ca0928-2f8b-4840-aa21-fb8e23d1865d-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c45fbd6df-49fzt\" (UID: \"a4ca0928-2f8b-4840-aa21-fb8e23d1865d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:28:35.680327 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.680025 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4ca0928-2f8b-4840-aa21-fb8e23d1865d-tmp\") pod \"klusterlet-addon-workmgr-6c45fbd6df-49fzt\" (UID: \"a4ca0928-2f8b-4840-aa21-fb8e23d1865d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:28:35.680387 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.680327 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4ca0928-2f8b-4840-aa21-fb8e23d1865d-tmp\") pod \"klusterlet-addon-workmgr-6c45fbd6df-49fzt\" (UID: \"a4ca0928-2f8b-4840-aa21-fb8e23d1865d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:28:35.682468 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.682444 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a4ca0928-2f8b-4840-aa21-fb8e23d1865d-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c45fbd6df-49fzt\" (UID: \"a4ca0928-2f8b-4840-aa21-fb8e23d1865d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:28:35.688210 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.688185 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rxcf\" (UniqueName: \"kubernetes.io/projected/a4ca0928-2f8b-4840-aa21-fb8e23d1865d-kube-api-access-7rxcf\") pod \"klusterlet-addon-workmgr-6c45fbd6df-49fzt\" (UID: \"a4ca0928-2f8b-4840-aa21-fb8e23d1865d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:28:35.821944 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.821860 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:28:35.939448 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:35.939414 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt"] Jul 30 02:28:35.942979 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:28:35.942954 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ca0928_2f8b_4840_aa21_fb8e23d1865d.slice/crio-14b20805de3f3c7ab68984ad0fdc7b285ed07413b73e14e6b77a48d7c26b5085 WatchSource:0}: Error finding container 14b20805de3f3c7ab68984ad0fdc7b285ed07413b73e14e6b77a48d7c26b5085: Status 404 returned error can't find the container with id 14b20805de3f3c7ab68984ad0fdc7b285ed07413b73e14e6b77a48d7c26b5085 Jul 30 02:28:36.304452 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:36.304422 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" event={"ID":"a4ca0928-2f8b-4840-aa21-fb8e23d1865d","Type":"ContainerStarted","Data":"14b20805de3f3c7ab68984ad0fdc7b285ed07413b73e14e6b77a48d7c26b5085"} Jul 30 02:28:41.318561 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:41.318520 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" event={"ID":"a4ca0928-2f8b-4840-aa21-fb8e23d1865d","Type":"ContainerStarted","Data":"456f57cbbb23014e1905a1c48e78c9b1025bd89cec6b09c0bc4bb86a4306f0a8"} Jul 30 02:28:41.319024 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:41.318730 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:28:41.320082 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:41.320060 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:28:41.335602 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:41.335564 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" podStartSLOduration=1.173310553 podStartE2EDuration="6.335551589s" totalImagesPullingTime="5.162241036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:28:41.334611495 +0000 UTC m=+169.013415494" watchObservedRunningTime="2026-07-30 02:28:41.335551589 +0000 UTC m=+169.014355584" Jul 30 02:28:43.846926 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:43.846878 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:28:43.846926 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:28:43.846931 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:29:43.441002 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:29:43.440961 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:04.595551 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:04.595508 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:30:04.596048 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:04.595681 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 30 02:30:04.596048 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:04.595764 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs podName:db23d664-ec37-4206-b1f3-ac52ecec65dc nodeName:}" failed. No retries permitted until 2026-07-30 02:32:06.595743428 +0000 UTC m=+374.274547418 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs") pod "network-metrics-daemon-rx95s" (UID: "db23d664-ec37-4206-b1f3-ac52ecec65dc") : secret "metrics-daemon-secret" not found Jul 30 02:30:10.210041 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.210003 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7lbt"] Jul 30 02:30:10.212076 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.212046 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7lbt" Jul 30 02:30:10.214739 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.214715 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 30 02:30:10.214872 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.214840 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 30 02:30:10.215835 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.215812 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ngbch\"" Jul 30 02:30:10.225492 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.225466 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7lbt"] Jul 30 02:30:10.234035 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.234011 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f42r5\" (UniqueName: \"kubernetes.io/projected/e0b136e6-cc5e-4f07-9e0b-ec7cfd534e84-kube-api-access-f42r5\") pod \"volume-data-source-validator-549c6d54cf-j7lbt\" (UID: \"e0b136e6-cc5e-4f07-9e0b-ec7cfd534e84\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7lbt" Jul 30 02:30:10.308209 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.308187 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-x9v2z"] Jul 30 02:30:10.310086 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.310072 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-x9v2z" Jul 30 02:30:10.312245 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.312226 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-x4h4w\"" Jul 30 02:30:10.314675 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.314643 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5"] Jul 30 02:30:10.316695 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.316675 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf"] Jul 30 02:30:10.316772 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.316737 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" Jul 30 02:30:10.318478 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.318463 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:10.319082 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.319063 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 30 02:30:10.319197 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.319182 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 30 02:30:10.319404 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.319391 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 30 02:30:10.319513 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.319489 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 30 02:30:10.319576 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.319494 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-t8f29\"" Jul 30 02:30:10.320651 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.320629 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-x9v2z"] Jul 30 02:30:10.320785 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.320749 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 30 02:30:10.320916 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.320899 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 30 02:30:10.320916 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.320909 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zxpgr\"" Jul 30 02:30:10.323218 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.323198 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 30 02:30:10.323315 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.323246 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 30 02:30:10.327452 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.327416 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5"] Jul 30 02:30:10.328559 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.328538 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf"] Jul 30 02:30:10.334406 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.334386 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f42r5\" (UniqueName: \"kubernetes.io/projected/e0b136e6-cc5e-4f07-9e0b-ec7cfd534e84-kube-api-access-f42r5\") pod \"volume-data-source-validator-549c6d54cf-j7lbt\" (UID: \"e0b136e6-cc5e-4f07-9e0b-ec7cfd534e84\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7lbt" Jul 30 02:30:10.334482 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.334416 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7rx8\" (UniqueName: \"kubernetes.io/projected/9c1fb660-1169-4e56-853c-8267f0a102e7-kube-api-access-t7rx8\") pod \"network-check-source-7d88cb55d8-x9v2z\" (UID: \"9c1fb660-1169-4e56-853c-8267f0a102e7\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-x9v2z" Jul 30 02:30:10.334482 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.334435 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sr2d\" (UniqueName: \"kubernetes.io/projected/a42dc3f7-c8d2-4e06-a0ea-69b8087a7920-kube-api-access-2sr2d\") pod \"service-ca-operator-5d6c75f84f-vbkt5\" (UID: \"a42dc3f7-c8d2-4e06-a0ea-69b8087a7920\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" Jul 30 02:30:10.334552 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.334529 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwpfh\" (UniqueName: \"kubernetes.io/projected/361c8b92-f91e-4d15-9167-383cb9ef1c90-kube-api-access-nwpfh\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:10.334552 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.334546 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a42dc3f7-c8d2-4e06-a0ea-69b8087a7920-config\") pod \"service-ca-operator-5d6c75f84f-vbkt5\" (UID: \"a42dc3f7-c8d2-4e06-a0ea-69b8087a7920\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" Jul 30 02:30:10.334613 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.334567 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:10.334613 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.334602 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/361c8b92-f91e-4d15-9167-383cb9ef1c90-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:10.334718 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.334704 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a42dc3f7-c8d2-4e06-a0ea-69b8087a7920-serving-cert\") pod \"service-ca-operator-5d6c75f84f-vbkt5\" (UID: \"a42dc3f7-c8d2-4e06-a0ea-69b8087a7920\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" Jul 30 02:30:10.350158 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.350132 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f42r5\" (UniqueName: \"kubernetes.io/projected/e0b136e6-cc5e-4f07-9e0b-ec7cfd534e84-kube-api-access-f42r5\") pod \"volume-data-source-validator-549c6d54cf-j7lbt\" (UID: \"e0b136e6-cc5e-4f07-9e0b-ec7cfd534e84\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7lbt" Jul 30 02:30:10.419146 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.419124 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:10.435080 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.435061 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7rx8\" (UniqueName: \"kubernetes.io/projected/9c1fb660-1169-4e56-853c-8267f0a102e7-kube-api-access-t7rx8\") pod \"network-check-source-7d88cb55d8-x9v2z\" (UID: \"9c1fb660-1169-4e56-853c-8267f0a102e7\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-x9v2z" Jul 30 02:30:10.435151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.435086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sr2d\" (UniqueName: \"kubernetes.io/projected/a42dc3f7-c8d2-4e06-a0ea-69b8087a7920-kube-api-access-2sr2d\") pod \"service-ca-operator-5d6c75f84f-vbkt5\" (UID: \"a42dc3f7-c8d2-4e06-a0ea-69b8087a7920\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" Jul 30 02:30:10.435151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.435113 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwpfh\" (UniqueName: \"kubernetes.io/projected/361c8b92-f91e-4d15-9167-383cb9ef1c90-kube-api-access-nwpfh\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:10.435151 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.435129 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a42dc3f7-c8d2-4e06-a0ea-69b8087a7920-config\") pod \"service-ca-operator-5d6c75f84f-vbkt5\" (UID: \"a42dc3f7-c8d2-4e06-a0ea-69b8087a7920\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" Jul 30 02:30:10.435311 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.435151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:10.435311 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.435195 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/361c8b92-f91e-4d15-9167-383cb9ef1c90-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:10.435311 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.435225 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a42dc3f7-c8d2-4e06-a0ea-69b8087a7920-serving-cert\") pod \"service-ca-operator-5d6c75f84f-vbkt5\" (UID: \"a42dc3f7-c8d2-4e06-a0ea-69b8087a7920\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" Jul 30 02:30:10.435311 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:10.435300 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 02:30:10.435501 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:10.435359 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls podName:361c8b92-f91e-4d15-9167-383cb9ef1c90 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:10.935340827 +0000 UTC m=+258.614144818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dsvcf" (UID: "361c8b92-f91e-4d15-9167-383cb9ef1c90") : secret "cluster-monitoring-operator-tls" not found Jul 30 02:30:10.435755 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.435736 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a42dc3f7-c8d2-4e06-a0ea-69b8087a7920-config\") pod \"service-ca-operator-5d6c75f84f-vbkt5\" (UID: \"a42dc3f7-c8d2-4e06-a0ea-69b8087a7920\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" Jul 30 02:30:10.435931 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.435916 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/361c8b92-f91e-4d15-9167-383cb9ef1c90-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:10.437372 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.437349 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a42dc3f7-c8d2-4e06-a0ea-69b8087a7920-serving-cert\") pod \"service-ca-operator-5d6c75f84f-vbkt5\" (UID: \"a42dc3f7-c8d2-4e06-a0ea-69b8087a7920\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" Jul 30 02:30:10.443031 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.443004 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sr2d\" (UniqueName: \"kubernetes.io/projected/a42dc3f7-c8d2-4e06-a0ea-69b8087a7920-kube-api-access-2sr2d\") pod \"service-ca-operator-5d6c75f84f-vbkt5\" (UID: \"a42dc3f7-c8d2-4e06-a0ea-69b8087a7920\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" Jul 30 02:30:10.443133 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.443049 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwpfh\" (UniqueName: \"kubernetes.io/projected/361c8b92-f91e-4d15-9167-383cb9ef1c90-kube-api-access-nwpfh\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:10.443327 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.443308 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7rx8\" (UniqueName: \"kubernetes.io/projected/9c1fb660-1169-4e56-853c-8267f0a102e7-kube-api-access-t7rx8\") pod \"network-check-source-7d88cb55d8-x9v2z\" (UID: \"9c1fb660-1169-4e56-853c-8267f0a102e7\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-x9v2z" Jul 30 02:30:10.485341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.485288 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:10.521337 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.521306 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7lbt" Jul 30 02:30:10.619009 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.618982 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-x9v2z" Jul 30 02:30:10.627376 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.627354 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" Jul 30 02:30:10.636941 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.636919 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7lbt"] Jul 30 02:30:10.639854 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:30:10.639823 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0b136e6_cc5e_4f07_9e0b_ec7cfd534e84.slice/crio-4705bfb0c6b09d4d069e6b68c1a853ec65af8940c34d047a7567c97e209da54c WatchSource:0}: Error finding container 4705bfb0c6b09d4d069e6b68c1a853ec65af8940c34d047a7567c97e209da54c: Status 404 returned error can't find the container with id 4705bfb0c6b09d4d069e6b68c1a853ec65af8940c34d047a7567c97e209da54c Jul 30 02:30:10.746656 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.746629 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-x9v2z"] Jul 30 02:30:10.749211 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:30:10.749182 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c1fb660_1169_4e56_853c_8267f0a102e7.slice/crio-359e06d9b83645c45e11cf08c0f4244053e540b3ad9b83e9c63c3246fd57bcdf WatchSource:0}: Error finding container 359e06d9b83645c45e11cf08c0f4244053e540b3ad9b83e9c63c3246fd57bcdf: Status 404 returned error can't find the container with id 359e06d9b83645c45e11cf08c0f4244053e540b3ad9b83e9c63c3246fd57bcdf Jul 30 02:30:10.760457 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.760423 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5"] Jul 30 02:30:10.766634 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:30:10.766613 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda42dc3f7_c8d2_4e06_a0ea_69b8087a7920.slice/crio-eacef4f7329abf1a4199dda12f9f4548b284153948a519429f2268356176cc0d WatchSource:0}: Error finding container eacef4f7329abf1a4199dda12f9f4548b284153948a519429f2268356176cc0d: Status 404 returned error can't find the container with id eacef4f7329abf1a4199dda12f9f4548b284153948a519429f2268356176cc0d Jul 30 02:30:10.938593 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:10.938562 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:10.938754 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:10.938707 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 02:30:10.938800 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:10.938766 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls podName:361c8b92-f91e-4d15-9167-383cb9ef1c90 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:11.938750731 +0000 UTC m=+259.617554704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dsvcf" (UID: "361c8b92-f91e-4d15-9167-383cb9ef1c90") : secret "cluster-monitoring-operator-tls" not found Jul 30 02:30:11.530858 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:11.530818 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" event={"ID":"a42dc3f7-c8d2-4e06-a0ea-69b8087a7920","Type":"ContainerStarted","Data":"eacef4f7329abf1a4199dda12f9f4548b284153948a519429f2268356176cc0d"} Jul 30 02:30:11.532947 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:11.532873 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-x9v2z" event={"ID":"9c1fb660-1169-4e56-853c-8267f0a102e7","Type":"ContainerStarted","Data":"29de4c4cea939f6c37df044153c9abbf8a69d4ee77fae5d43fe0fb570ffb68ac"} Jul 30 02:30:11.532947 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:11.532919 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-x9v2z" event={"ID":"9c1fb660-1169-4e56-853c-8267f0a102e7","Type":"ContainerStarted","Data":"359e06d9b83645c45e11cf08c0f4244053e540b3ad9b83e9c63c3246fd57bcdf"} Jul 30 02:30:11.534775 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:11.534738 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7lbt" event={"ID":"e0b136e6-cc5e-4f07-9e0b-ec7cfd534e84","Type":"ContainerStarted","Data":"4705bfb0c6b09d4d069e6b68c1a853ec65af8940c34d047a7567c97e209da54c"} Jul 30 02:30:11.547783 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:11.547735 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-x9v2z" podStartSLOduration=1.54771997 podStartE2EDuration="1.54771997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:30:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:30:11.54720318 +0000 UTC m=+259.226007175" watchObservedRunningTime="2026-07-30 02:30:11.54771997 +0000 UTC m=+259.226523965" Jul 30 02:30:11.947426 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:11.947337 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:11.947579 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:11.947491 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 02:30:11.947579 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:11.947569 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls podName:361c8b92-f91e-4d15-9167-383cb9ef1c90 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:13.947547883 +0000 UTC m=+261.626351862 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dsvcf" (UID: "361c8b92-f91e-4d15-9167-383cb9ef1c90") : secret "cluster-monitoring-operator-tls" not found Jul 30 02:30:12.540311 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:12.540266 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7lbt" event={"ID":"e0b136e6-cc5e-4f07-9e0b-ec7cfd534e84","Type":"ContainerStarted","Data":"758dc2b85e4e103142c3f391bdd4d5bc5232927f48bb390055b2ec84127f71e1"} Jul 30 02:30:12.556258 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:12.556202 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-j7lbt" podStartSLOduration=1.222504376 podStartE2EDuration="2.556186268s" totalImagesPullingTime="1.333681892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:30:12.555123601 +0000 UTC m=+260.233927597" watchObservedRunningTime="2026-07-30 02:30:12.556186268 +0000 UTC m=+260.234990263" Jul 30 02:30:12.632374 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:12.632339 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:12.689482 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:12.689369 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:13.545121 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:13.545074 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" event={"ID":"a42dc3f7-c8d2-4e06-a0ea-69b8087a7920","Type":"ContainerStarted","Data":"24193e178338f3187179f74ec64b35b9b4e2199a01ba4d8327288acdeea1a3ba"} Jul 30 02:30:13.561842 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:13.561792 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" podStartSLOduration=1.295203194 podStartE2EDuration="3.561771853s" totalImagesPullingTime="2.266568659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:30:13.559848299 +0000 UTC m=+261.238652293" watchObservedRunningTime="2026-07-30 02:30:13.561771853 +0000 UTC m=+261.240575848" Jul 30 02:30:13.872086 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:13.872009 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:13.960500 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:13.960469 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:13.963406 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:13.963379 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:13.963523 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:13.963508 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 02:30:13.963585 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:13.963576 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls podName:361c8b92-f91e-4d15-9167-383cb9ef1c90 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:17.963560362 +0000 UTC m=+265.642364339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dsvcf" (UID: "361c8b92-f91e-4d15-9167-383cb9ef1c90") : secret "cluster-monitoring-operator-tls" not found Jul 30 02:30:14.565510 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.565427 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:14.631050 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.631019 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2rwzh"] Jul 30 02:30:14.633916 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.633900 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:14.636447 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.636429 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 30 02:30:14.636698 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.636681 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 30 02:30:14.637517 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.637504 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6sh2l\"" Jul 30 02:30:14.637605 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.637582 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 30 02:30:14.637646 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.637626 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 30 02:30:14.642342 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.642319 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2rwzh"] Jul 30 02:30:14.646283 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.646260 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:14.670418 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.670394 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-data-volume\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:14.670516 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.670421 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:14.670516 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.670440 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-crio-socket\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:14.670606 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.670526 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:14.670720 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.670702 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6p68\" (UniqueName: \"kubernetes.io/projected/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-api-access-g6p68\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:14.703687 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.703645 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:14.771658 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.771636 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:14.771785 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.771713 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6p68\" (UniqueName: \"kubernetes.io/projected/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-api-access-g6p68\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:14.771785 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.771736 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-data-volume\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:14.771785 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.771755 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:14.771785 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.771775 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-crio-socket\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:14.771981 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:14.771792 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 02:30:14.771981 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:14.771838 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 02:30:14.771981 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.771849 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-crio-socket\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:14.771981 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:14.771869 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls podName:5452b49a-0c09-48bb-8cd3-1d4b19ce9473 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:15.271849591 +0000 UTC m=+262.950653568 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2rwzh" (UID: "5452b49a-0c09-48bb-8cd3-1d4b19ce9473") : secret "insights-runtime-extractor-tls" not found Jul 30 02:30:14.771981 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:14.771901 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm podName:5452b49a-0c09-48bb-8cd3-1d4b19ce9473 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:15.271888706 +0000 UTC m=+262.950692679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2rwzh" (UID: "5452b49a-0c09-48bb-8cd3-1d4b19ce9473") : configmap "kube-rbac-proxy" not found Jul 30 02:30:14.772196 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.772026 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-data-volume\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:14.779975 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:14.779960 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6p68\" (UniqueName: \"kubernetes.io/projected/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-api-access-g6p68\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:15.275605 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.275566 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:15.275790 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.275649 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:15.275790 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:15.275703 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 02:30:15.275790 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:15.275774 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm podName:5452b49a-0c09-48bb-8cd3-1d4b19ce9473 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:16.275757631 +0000 UTC m=+263.954561649 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2rwzh" (UID: "5452b49a-0c09-48bb-8cd3-1d4b19ce9473") : configmap "kube-rbac-proxy" not found Jul 30 02:30:15.275935 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:15.275792 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 02:30:15.275935 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:15.275850 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls podName:5452b49a-0c09-48bb-8cd3-1d4b19ce9473 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:16.275836848 +0000 UTC m=+263.954640821 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2rwzh" (UID: "5452b49a-0c09-48bb-8cd3-1d4b19ce9473") : secret "insights-runtime-extractor-tls" not found Jul 30 02:30:15.279758 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.279734 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-xsv25"] Jul 30 02:30:15.282168 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.282145 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:15.283019 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.283003 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xsv25" Jul 30 02:30:15.285499 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.285478 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-t4ksh\"" Jul 30 02:30:15.285596 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.285548 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 30 02:30:15.285596 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.285547 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 30 02:30:15.291451 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.291430 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-xsv25"] Jul 30 02:30:15.372525 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.372487 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:15.376171 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.376147 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l5rr\" (UniqueName: \"kubernetes.io/projected/c1b537ec-1ee0-4ac5-ae12-cf7d38acad6f-kube-api-access-7l5rr\") pod \"migrator-6865c9967-xsv25\" (UID: \"c1b537ec-1ee0-4ac5-ae12-cf7d38acad6f\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xsv25" Jul 30 02:30:15.435106 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.435081 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:15.476915 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.476887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l5rr\" (UniqueName: \"kubernetes.io/projected/c1b537ec-1ee0-4ac5-ae12-cf7d38acad6f-kube-api-access-7l5rr\") pod \"migrator-6865c9967-xsv25\" (UID: \"c1b537ec-1ee0-4ac5-ae12-cf7d38acad6f\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xsv25" Jul 30 02:30:15.484823 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.484804 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l5rr\" (UniqueName: \"kubernetes.io/projected/c1b537ec-1ee0-4ac5-ae12-cf7d38acad6f-kube-api-access-7l5rr\") pod \"migrator-6865c9967-xsv25\" (UID: \"c1b537ec-1ee0-4ac5-ae12-cf7d38acad6f\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xsv25" Jul 30 02:30:15.592761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.592689 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xsv25" Jul 30 02:30:15.717791 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:15.717761 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-xsv25"] Jul 30 02:30:15.720179 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:30:15.720154 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b537ec_1ee0_4ac5_ae12_cf7d38acad6f.slice/crio-d3f911f6cb29929fac1e16ecc2bd9798fbfbffa809c6c28342b97365316b9ca8 WatchSource:0}: Error finding container d3f911f6cb29929fac1e16ecc2bd9798fbfbffa809c6c28342b97365316b9ca8: Status 404 returned error can't find the container with id d3f911f6cb29929fac1e16ecc2bd9798fbfbffa809c6c28342b97365316b9ca8 Jul 30 02:30:16.195386 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:16.195354 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:16.283908 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:16.283877 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:16.284030 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:16.283947 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:16.284030 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:16.283989 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 02:30:16.284109 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:16.284053 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm podName:5452b49a-0c09-48bb-8cd3-1d4b19ce9473 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:18.284035195 +0000 UTC m=+265.962839172 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2rwzh" (UID: "5452b49a-0c09-48bb-8cd3-1d4b19ce9473") : configmap "kube-rbac-proxy" not found Jul 30 02:30:16.284109 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:16.284068 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 02:30:16.284189 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:16.284123 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls podName:5452b49a-0c09-48bb-8cd3-1d4b19ce9473 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:18.284109542 +0000 UTC m=+265.962913519 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2rwzh" (UID: "5452b49a-0c09-48bb-8cd3-1d4b19ce9473") : secret "insights-runtime-extractor-tls" not found Jul 30 02:30:16.415139 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:16.415112 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-84dj8_2a06f56e-3de8-45af-b3aa-da354ff4e895/dns-node-resolver/0.log" Jul 30 02:30:16.556144 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:16.556099 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xsv25" event={"ID":"c1b537ec-1ee0-4ac5-ae12-cf7d38acad6f","Type":"ContainerStarted","Data":"d3f911f6cb29929fac1e16ecc2bd9798fbfbffa809c6c28342b97365316b9ca8"} Jul 30 02:30:17.080164 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:17.080122 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:17.560642 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:17.560605 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xsv25" event={"ID":"c1b537ec-1ee0-4ac5-ae12-cf7d38acad6f","Type":"ContainerStarted","Data":"e5202b492bea575a8cfb852e931a77d45c164df97a7627a4ad54eaa89851d2ff"} Jul 30 02:30:17.560826 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:17.560686 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xsv25" event={"ID":"c1b537ec-1ee0-4ac5-ae12-cf7d38acad6f","Type":"ContainerStarted","Data":"bf5765eeccb659aa02568764789352cf983f05dacc75163b331fe643aa04dba2"} Jul 30 02:30:17.575635 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:17.575584 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-xsv25" podStartSLOduration=1.120998954 podStartE2EDuration="2.575566529s" totalImagesPullingTime="1.454567575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:30:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:30:17.575555504 +0000 UTC m=+265.254359502" watchObservedRunningTime="2026-07-30 02:30:17.575566529 +0000 UTC m=+265.254370525" Jul 30 02:30:17.816384 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:17.816312 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tsrrd_735cd8d6-2250-4b53-abef-118129c12350/node-ca/0.log" Jul 30 02:30:17.999611 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:17.999574 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:17.999782 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:17.999761 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 02:30:17.999843 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:17.999834 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls podName:361c8b92-f91e-4d15-9167-383cb9ef1c90 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:25.999817518 +0000 UTC m=+273.678621490 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dsvcf" (UID: "361c8b92-f91e-4d15-9167-383cb9ef1c90") : secret "cluster-monitoring-operator-tls" not found Jul 30 02:30:18.302338 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:18.302300 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:18.302697 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:18.302369 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:18.302697 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:18.302417 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 02:30:18.302697 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:18.302456 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 02:30:18.302697 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:18.302482 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm podName:5452b49a-0c09-48bb-8cd3-1d4b19ce9473 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:22.302465875 +0000 UTC m=+269.981269868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2rwzh" (UID: "5452b49a-0c09-48bb-8cd3-1d4b19ce9473") : configmap "kube-rbac-proxy" not found Jul 30 02:30:18.302697 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:18.302497 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls podName:5452b49a-0c09-48bb-8cd3-1d4b19ce9473 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:22.302491047 +0000 UTC m=+269.981295020 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2rwzh" (UID: "5452b49a-0c09-48bb-8cd3-1d4b19ce9473") : secret "insights-runtime-extractor-tls" not found Jul 30 02:30:18.414923 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:18.414898 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7pj54_b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0/serve-healthcheck-canary/0.log" Jul 30 02:30:19.429966 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:19.429925 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:19.706656 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:19.706563 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:19.911677 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:19.911642 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:20.022868 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:20.022837 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:20.085002 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:20.084970 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:20.146381 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:20.146350 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:20.612297 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:20.612265 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:21.212710 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:21.212675 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:21.605033 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:21.605002 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:22.014896 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:22.014867 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:22.336773 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:22.336681 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:22.336928 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:22.336807 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 02:30:22.336928 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:22.336832 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:22.336928 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:22.336871 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm podName:5452b49a-0c09-48bb-8cd3-1d4b19ce9473 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:30.336856302 +0000 UTC m=+278.015660274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2rwzh" (UID: "5452b49a-0c09-48bb-8cd3-1d4b19ce9473") : configmap "kube-rbac-proxy" not found Jul 30 02:30:22.336928 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:22.336924 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 02:30:22.337085 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:22.336970 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls podName:5452b49a-0c09-48bb-8cd3-1d4b19ce9473 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:30.336958824 +0000 UTC m=+278.015762797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2rwzh" (UID: "5452b49a-0c09-48bb-8cd3-1d4b19ce9473") : secret "insights-runtime-extractor-tls" not found Jul 30 02:30:22.412175 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:22.412142 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:22.613343 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:22.613255 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:23.208564 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:23.208533 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:23.404966 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:23.404932 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:23.598535 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:23.598508 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:23.913154 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:23.913060 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:24.016956 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:24.016925 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:24.602872 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:24.602826 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:24.807999 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:24.807963 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:25.212777 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:25.212741 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:25.411476 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:25.411448 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:25.802943 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:25.802895 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:26.068718 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:26.068617 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:26.068878 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:26.068784 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 02:30:26.068878 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:26.068857 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls podName:361c8b92-f91e-4d15-9167-383cb9ef1c90 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:42.068839413 +0000 UTC m=+289.747643385 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dsvcf" (UID: "361c8b92-f91e-4d15-9167-383cb9ef1c90") : secret "cluster-monitoring-operator-tls" not found Jul 30 02:30:26.198610 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:26.198579 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:26.405359 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:26.405273 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:26.807021 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:26.806990 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:27.005512 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:27.005480 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:27.119592 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:27.119505 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:27.601231 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:27.601175 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:28.004271 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:28.004241 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:28.407702 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:28.407606 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:28.808852 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:28.808821 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:28.909646 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:28.909612 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:29.216179 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:29.216103 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:29.406794 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:29.406762 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:29.824579 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:29.824547 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:30.228309 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:30.228230 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:30.400958 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:30.400930 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:30.401077 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:30.400992 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:30.401513 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:30.401498 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:30.403236 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:30.403217 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5452b49a-0c09-48bb-8cd3-1d4b19ce9473-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2rwzh\" (UID: \"5452b49a-0c09-48bb-8cd3-1d4b19ce9473\") " pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:30.405989 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:30.405955 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:30.543050 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:30.543028 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2rwzh" Jul 30 02:30:30.665647 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:30.665621 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2rwzh"] Jul 30 02:30:30.667943 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:30:30.667915 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5452b49a_0c09_48bb_8cd3_1d4b19ce9473.slice/crio-2cc20d4292c38b430472106cc5fbe8cbf0dd046af652134f3239a6de66e3b40a WatchSource:0}: Error finding container 2cc20d4292c38b430472106cc5fbe8cbf0dd046af652134f3239a6de66e3b40a: Status 404 returned error can't find the container with id 2cc20d4292c38b430472106cc5fbe8cbf0dd046af652134f3239a6de66e3b40a Jul 30 02:30:31.605234 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:31.605193 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2rwzh" event={"ID":"5452b49a-0c09-48bb-8cd3-1d4b19ce9473","Type":"ContainerStarted","Data":"2cc20d4292c38b430472106cc5fbe8cbf0dd046af652134f3239a6de66e3b40a"} Jul 30 02:30:33.289628 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:33.289588 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7rjk5" podUID="d67b207e-c987-490a-9051-3006d810dee3" Jul 30 02:30:33.289628 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:33.289590 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" podUID="1b389afb-4064-40ac-a473-89da51b1f974" Jul 30 02:30:33.612436 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:33.612359 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:30:33.612571 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:33.612371 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7rjk5" Jul 30 02:30:34.912534 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:34.912490 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:35.011222 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:35.011192 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:35.192190 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:35.192073 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:35.290062 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:35.290024 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:36.625314 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:36.625281 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2rwzh" event={"ID":"5452b49a-0c09-48bb-8cd3-1d4b19ce9473","Type":"ContainerStarted","Data":"5fc0a6da9cca6ae370c43c031abbaa2bfea4a2358b375776525e5276ba46b2dc"} Jul 30 02:30:37.156324 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:37.156282 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:30:37.158944 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:37.158921 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b389afb-4064-40ac-a473-89da51b1f974-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-rr58w\" (UID: \"1b389afb-4064-40ac-a473-89da51b1f974\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:30:37.216066 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:37.216028 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-62dmk\"" Jul 30 02:30:37.224000 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:37.223974 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" Jul 30 02:30:37.257627 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:37.257585 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:30:37.257823 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:37.257717 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:30:37.260383 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:37.260354 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d67b207e-c987-490a-9051-3006d810dee3-metrics-tls\") pod \"dns-default-7rjk5\" (UID: \"d67b207e-c987-490a-9051-3006d810dee3\") " pod="openshift-dns/dns-default-7rjk5" Jul 30 02:30:37.261081 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:37.261046 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") pod \"image-registry-6c7765b4fc-p4gvx\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:30:37.516051 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:37.516019 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5sqnz\"" Jul 30 02:30:37.523957 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:37.523926 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7rjk5" Jul 30 02:30:37.550790 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:37.550762 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6r6b9\"" Jul 30 02:30:37.558589 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:37.558561 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:30:39.092805 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:39.092699 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7rjk5"] Jul 30 02:30:39.095200 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:30:39.095175 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd67b207e_c987_490a_9051_3006d810dee3.slice/crio-d71d91dc4330fd8c25913cabd01f9049c8efd1edc628991943f0265da3c601c6 WatchSource:0}: Error finding container d71d91dc4330fd8c25913cabd01f9049c8efd1edc628991943f0265da3c601c6: Status 404 returned error can't find the container with id d71d91dc4330fd8c25913cabd01f9049c8efd1edc628991943f0265da3c601c6 Jul 30 02:30:39.299912 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:39.299877 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-rr58w"] Jul 30 02:30:39.303052 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:30:39.303020 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b389afb_4064_40ac_a473_89da51b1f974.slice/crio-9d1bef0d8ab871a75d1ff02e4f013915aa5d40e1047d54fb8953c15edc882080 WatchSource:0}: Error finding container 9d1bef0d8ab871a75d1ff02e4f013915aa5d40e1047d54fb8953c15edc882080: Status 404 returned error can't find the container with id 9d1bef0d8ab871a75d1ff02e4f013915aa5d40e1047d54fb8953c15edc882080 Jul 30 02:30:39.303240 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:39.303224 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c7765b4fc-p4gvx"] Jul 30 02:30:39.305867 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:30:39.305846 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca64202a_9a1b_42d4_b980_cabdcddc4976.slice/crio-5be63e733fae273616bfd2ef95912b0e87fded6e33e87397d7ee91c40f0aafeb WatchSource:0}: Error finding container 5be63e733fae273616bfd2ef95912b0e87fded6e33e87397d7ee91c40f0aafeb: Status 404 returned error can't find the container with id 5be63e733fae273616bfd2ef95912b0e87fded6e33e87397d7ee91c40f0aafeb Jul 30 02:30:39.642442 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:39.642318 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7rjk5" event={"ID":"d67b207e-c987-490a-9051-3006d810dee3","Type":"ContainerStarted","Data":"d71d91dc4330fd8c25913cabd01f9049c8efd1edc628991943f0265da3c601c6"} Jul 30 02:30:39.645422 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:39.645331 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" event={"ID":"ca64202a-9a1b-42d4-b980-cabdcddc4976","Type":"ContainerStarted","Data":"09548b83edc4ef6ba9bd09855af6e4a27d56226b90f92125540d5e4bcfa66c38"} Jul 30 02:30:39.645422 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:39.645387 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" event={"ID":"ca64202a-9a1b-42d4-b980-cabdcddc4976","Type":"ContainerStarted","Data":"5be63e733fae273616bfd2ef95912b0e87fded6e33e87397d7ee91c40f0aafeb"} Jul 30 02:30:39.645611 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:39.645553 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:30:39.647857 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:39.647832 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2rwzh" event={"ID":"5452b49a-0c09-48bb-8cd3-1d4b19ce9473","Type":"ContainerStarted","Data":"03ac04487bb5c7950a63c25216e767ffce199274fcf0c1919d4d98100c1c9dc0"} Jul 30 02:30:39.649217 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:39.649192 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" event={"ID":"1b389afb-4064-40ac-a473-89da51b1f974","Type":"ContainerStarted","Data":"9d1bef0d8ab871a75d1ff02e4f013915aa5d40e1047d54fb8953c15edc882080"} Jul 30 02:30:39.666156 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:39.666105 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" podStartSLOduration=286.666085818 podStartE2EDuration="4m46.666085818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:25:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:30:39.664704612 +0000 UTC m=+287.343508843" watchObservedRunningTime="2026-07-30 02:30:39.666085818 +0000 UTC m=+287.344889795" Jul 30 02:30:41.319218 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:41.319164 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" podUID="a4ca0928-2f8b-4840-aa21-fb8e23d1865d" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.10:8000/readyz\": dial tcp 10.132.0.10:8000: connect: connection refused" Jul 30 02:30:42.096601 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:42.096553 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:42.099249 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:42.099216 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/361c8b92-f91e-4d15-9167-383cb9ef1c90-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dsvcf\" (UID: \"361c8b92-f91e-4d15-9167-383cb9ef1c90\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:42.133061 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:42.133029 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" Jul 30 02:30:42.993045 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:42.993016 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf"] Jul 30 02:30:42.996419 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:30:42.996390 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod361c8b92_f91e_4d15_9167_383cb9ef1c90.slice/crio-5b6ebd05de6763882af8833a3b3cfb51233769a5c0c5dcec7635fba5918ea364 WatchSource:0}: Error finding container 5b6ebd05de6763882af8833a3b3cfb51233769a5c0c5dcec7635fba5918ea364: Status 404 returned error can't find the container with id 5b6ebd05de6763882af8833a3b3cfb51233769a5c0c5dcec7635fba5918ea364 Jul 30 02:30:43.666088 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:43.666043 2566 generic.go:356] "Generic (PLEG): container finished" podID="a4ca0928-2f8b-4840-aa21-fb8e23d1865d" containerID="456f57cbbb23014e1905a1c48e78c9b1025bd89cec6b09c0bc4bb86a4306f0a8" exitCode=1 Jul 30 02:30:43.666281 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:43.666127 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" event={"ID":"a4ca0928-2f8b-4840-aa21-fb8e23d1865d","Type":"ContainerDied","Data":"456f57cbbb23014e1905a1c48e78c9b1025bd89cec6b09c0bc4bb86a4306f0a8"} Jul 30 02:30:43.666513 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:43.666495 2566 scope.go:117] "RemoveContainer" containerID="456f57cbbb23014e1905a1c48e78c9b1025bd89cec6b09c0bc4bb86a4306f0a8" Jul 30 02:30:43.667888 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:43.667860 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7rjk5" event={"ID":"d67b207e-c987-490a-9051-3006d810dee3","Type":"ContainerStarted","Data":"c69121e830b3ea59282c5f817f9719b0983978d9331811e4b0d55b0dbc7b6b38"} Jul 30 02:30:43.667997 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:43.667916 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7rjk5" event={"ID":"d67b207e-c987-490a-9051-3006d810dee3","Type":"ContainerStarted","Data":"e213ed4b4ad7517c7ac8d0e8b4ad44955e6b279ad25d70f21e64ed2c53330f81"} Jul 30 02:30:43.667997 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:43.667949 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7rjk5" Jul 30 02:30:43.669899 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:43.669878 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2rwzh" event={"ID":"5452b49a-0c09-48bb-8cd3-1d4b19ce9473","Type":"ContainerStarted","Data":"674d1968a0ede7d93673c4e4fbb75b08940766cbae691c29a730b3c71baf9cdd"} Jul 30 02:30:43.671371 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:43.671349 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" event={"ID":"1b389afb-4064-40ac-a473-89da51b1f974","Type":"ContainerStarted","Data":"3237e7537b82ce0425a5b3a2286e343734777af18ac061ceca68b5aad9206b81"} Jul 30 02:30:43.672488 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:43.672468 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" event={"ID":"361c8b92-f91e-4d15-9167-383cb9ef1c90","Type":"ContainerStarted","Data":"5b6ebd05de6763882af8833a3b3cfb51233769a5c0c5dcec7635fba5918ea364"} Jul 30 02:30:43.698784 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:43.698724 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-rr58w" podStartSLOduration=273.166707032 podStartE2EDuration="4m36.69870379s" totalImagesPullingTime="3.531996758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:26:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:30:43.698318639 +0000 UTC m=+291.377122635" watchObservedRunningTime="2026-07-30 02:30:43.69870379 +0000 UTC m=+291.377507787" Jul 30 02:30:43.715814 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:43.715751 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2rwzh" podStartSLOduration=17.661031498 podStartE2EDuration="29.715728767s" totalImagesPullingTime="12.054697269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 02:30:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:30:43.714774695 +0000 UTC m=+291.393578703" watchObservedRunningTime="2026-07-30 02:30:43.715728767 +0000 UTC m=+291.394532763" Jul 30 02:30:43.732975 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:43.732932 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7rjk5" podStartSLOduration=252.990654314 podStartE2EDuration="4m16.732917113s" totalImagesPullingTime="3.742262799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:30:43.732576441 +0000 UTC m=+291.411380437" watchObservedRunningTime="2026-07-30 02:30:43.732917113 +0000 UTC m=+291.411721109" Jul 30 02:30:43.799100 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:43.799068 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:44.679634 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:44.679588 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" event={"ID":"a4ca0928-2f8b-4840-aa21-fb8e23d1865d","Type":"ContainerStarted","Data":"f988ea8cdcb378dd7661beab7b2462279fd51662d867472dd72c2691f6d0ef8c"} Jul 30 02:30:44.680307 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:44.680277 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:30:44.681015 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:44.680995 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c45fbd6df-49fzt" Jul 30 02:30:45.331267 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:45.331238 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:45.432061 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:45.432029 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:45.684492 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:45.684413 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" event={"ID":"361c8b92-f91e-4d15-9167-383cb9ef1c90","Type":"ContainerStarted","Data":"02baa377bd76c721ef5f8e81605c98639763ba8d3f00f289050a4f8de7d36b3e"} Jul 30 02:30:45.700564 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:45.700521 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dsvcf" podStartSLOduration=33.872488516 podStartE2EDuration="35.700506199s" totalImagesPullingTime="1.828017683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:30:45.699409152 +0000 UTC m=+293.378213150" watchObservedRunningTime="2026-07-30 02:30:45.700506199 +0000 UTC m=+293.379310193" Jul 30 02:30:46.848280 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:46.848245 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rx95s" podUID="db23d664-ec37-4206-b1f3-ac52ecec65dc" Jul 30 02:30:48.325343 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.325306 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:48.356299 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.356268 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-7hf6p"] Jul 30 02:30:48.358924 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.358907 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:48.361184 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.361162 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 30 02:30:48.362132 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.362112 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-6jjwd\"" Jul 30 02:30:48.362132 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.362118 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 30 02:30:48.362284 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.362146 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 30 02:30:48.368892 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.368868 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-7hf6p"] Jul 30 02:30:48.387896 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.387866 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:48.450021 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.449985 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-7hf6p\" (UID: \"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:48.450110 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.450042 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-7hf6p\" (UID: \"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:48.450164 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.450129 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8mbx\" (UniqueName: \"kubernetes.io/projected/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-kube-api-access-v8mbx\") pod \"prometheus-operator-84d69f794d-7hf6p\" (UID: \"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:48.450216 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.450198 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-7hf6p\" (UID: \"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:48.551497 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.551457 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-7hf6p\" (UID: \"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:48.551634 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.551506 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8mbx\" (UniqueName: \"kubernetes.io/projected/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-kube-api-access-v8mbx\") pod \"prometheus-operator-84d69f794d-7hf6p\" (UID: \"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:48.551634 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.551544 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-7hf6p\" (UID: \"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:48.551634 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.551576 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-7hf6p\" (UID: \"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:48.551754 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:48.551693 2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 30 02:30:48.551787 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:30:48.551757 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-prometheus-operator-tls podName:ee72ae37-8b70-41f4-9fe5-96c3e4bb4291 nodeName:}" failed. No retries permitted until 2026-07-30 02:30:49.051741653 +0000 UTC m=+296.730545625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-7hf6p" (UID: "ee72ae37-8b70-41f4-9fe5-96c3e4bb4291") : secret "prometheus-operator-tls" not found Jul 30 02:30:48.552278 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.552257 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-7hf6p\" (UID: \"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:48.553926 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.553900 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-7hf6p\" (UID: \"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:48.560937 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.560915 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8mbx\" (UniqueName: \"kubernetes.io/projected/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-kube-api-access-v8mbx\") pod \"prometheus-operator-84d69f794d-7hf6p\" (UID: \"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:48.660847 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:48.660755 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:49.056136 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:49.056099 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-7hf6p\" (UID: \"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:49.058491 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:49.058460 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ee72ae37-8b70-41f4-9fe5-96c3e4bb4291-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-7hf6p\" (UID: \"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:49.268535 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:49.268497 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" Jul 30 02:30:49.390027 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:49.389886 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-7hf6p"] Jul 30 02:30:49.392304 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:30:49.392281 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee72ae37_8b70_41f4_9fe5_96c3e4bb4291.slice/crio-ab89b527bc5859c24311fbee9d079bb8c7f83b6c8da3868f3a03b4576682634a WatchSource:0}: Error finding container ab89b527bc5859c24311fbee9d079bb8c7f83b6c8da3868f3a03b4576682634a: Status 404 returned error can't find the container with id ab89b527bc5859c24311fbee9d079bb8c7f83b6c8da3868f3a03b4576682634a Jul 30 02:30:49.698209 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:49.698131 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" event={"ID":"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291","Type":"ContainerStarted","Data":"ab89b527bc5859c24311fbee9d079bb8c7f83b6c8da3868f3a03b4576682634a"} Jul 30 02:30:50.185156 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:50.185119 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:50.243812 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:50.243780 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:50.946062 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:50.945950 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:51.706220 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:51.706191 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" event={"ID":"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291","Type":"ContainerStarted","Data":"8ebe57cc351b0e9a1de4b36ba9dd721a8da203ac0b640ada6df9cbe950e48801"} Jul 30 02:30:51.706390 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:51.706242 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" event={"ID":"ee72ae37-8b70-41f4-9fe5-96c3e4bb4291","Type":"ContainerStarted","Data":"01e003f24c8ce6cbad6071c58630f3abf310faa7e1680d9846eb501e137447a5"} Jul 30 02:30:51.722274 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:51.722233 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-7hf6p" podStartSLOduration=2.443005448 podStartE2EDuration="3.722217568s" totalImagesPullingTime="1.27921212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:30:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:30:51.720578229 +0000 UTC m=+299.399382222" watchObservedRunningTime="2026-07-30 02:30:51.722217568 +0000 UTC m=+299.401021563" Jul 30 02:30:52.741572 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:52.741544 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 30 02:30:53.453571 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.453531 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:53.666538 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.666502 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:53.681334 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.681309 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7rjk5" Jul 30 02:30:53.687845 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.687825 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd"] Jul 30 02:30:53.690873 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.690860 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:53.693201 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.693180 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dg2nz\"" Jul 30 02:30:53.693288 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.693235 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 30 02:30:53.693698 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.693680 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 30 02:30:53.700924 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.700904 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd"] Jul 30 02:30:53.705627 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.705581 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t7kcf"] Jul 30 02:30:53.712179 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.712157 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.714261 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.714241 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 30 02:30:53.714691 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.714675 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-h2qjz\"" Jul 30 02:30:53.714942 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.714928 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 30 02:30:53.715122 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.715109 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 30 02:30:53.764851 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.764784 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:53.793720 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.793692 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97bc4540-f598-47e6-9a59-da24d7e49ec7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-882jd\" (UID: \"97bc4540-f598-47e6-9a59-da24d7e49ec7\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:53.793875 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.793756 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6z72\" (UniqueName: \"kubernetes.io/projected/97bc4540-f598-47e6-9a59-da24d7e49ec7-kube-api-access-s6z72\") pod \"openshift-state-metrics-b8f49f5c6-882jd\" (UID: \"97bc4540-f598-47e6-9a59-da24d7e49ec7\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:53.793875 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.793816 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97bc4540-f598-47e6-9a59-da24d7e49ec7-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-882jd\" (UID: \"97bc4540-f598-47e6-9a59-da24d7e49ec7\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:53.793993 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.793900 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97bc4540-f598-47e6-9a59-da24d7e49ec7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-882jd\" (UID: \"97bc4540-f598-47e6-9a59-da24d7e49ec7\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:53.825582 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.825547 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:53.894330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.894302 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97bc4540-f598-47e6-9a59-da24d7e49ec7-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-882jd\" (UID: \"97bc4540-f598-47e6-9a59-da24d7e49ec7\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:53.894468 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.894335 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.894468 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.894358 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfcf4fa5-3952-4914-a9db-013f7be543b2-sys\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.894468 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.894376 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97bc4540-f598-47e6-9a59-da24d7e49ec7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-882jd\" (UID: \"97bc4540-f598-47e6-9a59-da24d7e49ec7\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:53.894468 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.894400 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97bc4540-f598-47e6-9a59-da24d7e49ec7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-882jd\" (UID: \"97bc4540-f598-47e6-9a59-da24d7e49ec7\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:53.894468 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.894424 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hhnc\" (UniqueName: \"kubernetes.io/projected/dfcf4fa5-3952-4914-a9db-013f7be543b2-kube-api-access-6hhnc\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.894468 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.894449 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dfcf4fa5-3952-4914-a9db-013f7be543b2-root\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.894739 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.894471 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfcf4fa5-3952-4914-a9db-013f7be543b2-metrics-client-ca\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.894739 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.894497 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-textfile\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.894739 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.894517 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6z72\" (UniqueName: \"kubernetes.io/projected/97bc4540-f598-47e6-9a59-da24d7e49ec7-kube-api-access-s6z72\") pod \"openshift-state-metrics-b8f49f5c6-882jd\" (UID: \"97bc4540-f598-47e6-9a59-da24d7e49ec7\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:53.894739 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.894573 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-accelerators-collector-config\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.894739 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.894593 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-wtmp\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.894739 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.894618 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-tls\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.895049 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.895029 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97bc4540-f598-47e6-9a59-da24d7e49ec7-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-882jd\" (UID: \"97bc4540-f598-47e6-9a59-da24d7e49ec7\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:53.896820 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.896799 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97bc4540-f598-47e6-9a59-da24d7e49ec7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-882jd\" (UID: \"97bc4540-f598-47e6-9a59-da24d7e49ec7\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:53.896911 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.896823 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97bc4540-f598-47e6-9a59-da24d7e49ec7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-882jd\" (UID: \"97bc4540-f598-47e6-9a59-da24d7e49ec7\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:53.905123 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.905093 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6z72\" (UniqueName: \"kubernetes.io/projected/97bc4540-f598-47e6-9a59-da24d7e49ec7-kube-api-access-s6z72\") pod \"openshift-state-metrics-b8f49f5c6-882jd\" (UID: \"97bc4540-f598-47e6-9a59-da24d7e49ec7\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:53.926527 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.926485 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:53.995179 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995112 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-textfile\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.995179 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995146 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-accelerators-collector-config\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.995179 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995166 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-wtmp\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.995400 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995191 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-tls\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.995400 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995369 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-wtmp\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.995554 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995429 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.995554 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfcf4fa5-3952-4914-a9db-013f7be543b2-sys\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.995554 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995518 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hhnc\" (UniqueName: \"kubernetes.io/projected/dfcf4fa5-3952-4914-a9db-013f7be543b2-kube-api-access-6hhnc\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.995554 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995514 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-textfile\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.995782 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995564 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfcf4fa5-3952-4914-a9db-013f7be543b2-sys\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.995782 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995640 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dfcf4fa5-3952-4914-a9db-013f7be543b2-root\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.995782 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995690 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfcf4fa5-3952-4914-a9db-013f7be543b2-metrics-client-ca\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.995782 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995716 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dfcf4fa5-3952-4914-a9db-013f7be543b2-root\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.995950 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.995872 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-accelerators-collector-config\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.996150 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.996128 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfcf4fa5-3952-4914-a9db-013f7be543b2-metrics-client-ca\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.997572 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.997551 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-tls\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.997923 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.997900 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfcf4fa5-3952-4914-a9db-013f7be543b2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:53.999754 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:53.999739 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" Jul 30 02:30:54.003110 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.003088 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hhnc\" (UniqueName: \"kubernetes.io/projected/dfcf4fa5-3952-4914-a9db-013f7be543b2-kube-api-access-6hhnc\") pod \"node-exporter-t7kcf\" (UID: \"dfcf4fa5-3952-4914-a9db-013f7be543b2\") " pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:54.021944 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.021925 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t7kcf" Jul 30 02:30:54.027257 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.027210 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:54.033803 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:30:54.033776 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfcf4fa5_3952_4914_a9db_013f7be543b2.slice/crio-052b291b11fc752c1de43f8f9d36e0f49762b63a5952a9c8d1e36732214eb80d WatchSource:0}: Error finding container 052b291b11fc752c1de43f8f9d36e0f49762b63a5952a9c8d1e36732214eb80d: Status 404 returned error can't find the container with id 052b291b11fc752c1de43f8f9d36e0f49762b63a5952a9c8d1e36732214eb80d Jul 30 02:30:54.035601 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.035582 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 02:30:54.133969 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.133945 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd"] Jul 30 02:30:54.136376 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:30:54.136352 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97bc4540_f598_47e6_9a59_da24d7e49ec7.slice/crio-04c65d3304bc13d386124ef33c74f49abc23e57b2da78c44ce825e2446613a71 WatchSource:0}: Error finding container 04c65d3304bc13d386124ef33c74f49abc23e57b2da78c44ce825e2446613a71: Status 404 returned error can't find the container with id 04c65d3304bc13d386124ef33c74f49abc23e57b2da78c44ce825e2446613a71 Jul 30 02:30:54.422759 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.422720 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:54.522554 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.522476 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:54.575188 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.575153 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:54.676200 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.676158 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:54.719042 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.719008 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" event={"ID":"97bc4540-f598-47e6-9a59-da24d7e49ec7","Type":"ContainerStarted","Data":"36a3a9411ba2f2cdd98cd02802bf3f480ff5952c50eceef08dc2781fa865f669"} Jul 30 02:30:54.719205 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.719074 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" event={"ID":"97bc4540-f598-47e6-9a59-da24d7e49ec7","Type":"ContainerStarted","Data":"b1c2c804cd6585540e8d15c65bc452f89571cc9be9a5934a20289f5e0f258371"} Jul 30 02:30:54.719205 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.719107 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" event={"ID":"97bc4540-f598-47e6-9a59-da24d7e49ec7","Type":"ContainerStarted","Data":"04c65d3304bc13d386124ef33c74f49abc23e57b2da78c44ce825e2446613a71"} Jul 30 02:30:54.721023 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.720980 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t7kcf" event={"ID":"dfcf4fa5-3952-4914-a9db-013f7be543b2","Type":"ContainerStarted","Data":"052b291b11fc752c1de43f8f9d36e0f49762b63a5952a9c8d1e36732214eb80d"} Jul 30 02:30:54.751454 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:54.751419 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:55.262767 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:55.262730 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:55.726771 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:55.726693 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" event={"ID":"97bc4540-f598-47e6-9a59-da24d7e49ec7","Type":"ContainerStarted","Data":"d537fc028621b7e6fcbab89c951b3d47e66ee8f01dba1dd47cbcc8414ad6a24d"} Jul 30 02:30:55.728430 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:55.728398 2566 generic.go:356] "Generic (PLEG): container finished" podID="dfcf4fa5-3952-4914-a9db-013f7be543b2" containerID="27541c88f90bc8602cb1ce1116330e82df6e1a7d26237776e13c5e6b830cbbba" exitCode=0 Jul 30 02:30:55.728553 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:55.728432 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t7kcf" event={"ID":"dfcf4fa5-3952-4914-a9db-013f7be543b2","Type":"ContainerDied","Data":"27541c88f90bc8602cb1ce1116330e82df6e1a7d26237776e13c5e6b830cbbba"} Jul 30 02:30:55.743478 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:55.743437 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-882jd" podStartSLOduration=1.664767414 podStartE2EDuration="2.743424237s" totalImagesPullingTime="1.078656823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:30:55.742761797 +0000 UTC m=+303.421565792" watchObservedRunningTime="2026-07-30 02:30:55.743424237 +0000 UTC m=+303.422228231" Jul 30 02:30:55.860438 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:55.860407 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:55.956804 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:55.956777 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:56.015213 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:56.015183 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:56.355784 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:56.355708 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:56.528859 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:56.528824 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:56.618749 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:56.618658 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:56.689506 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:56.689469 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:56.738935 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:56.738902 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t7kcf" event={"ID":"dfcf4fa5-3952-4914-a9db-013f7be543b2","Type":"ContainerStarted","Data":"7e7ffdeba424a19670c9594c62b5ee2e2a3b8249f74b7194a80bf36c76c00b27"} Jul 30 02:30:56.739079 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:56.738980 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t7kcf" event={"ID":"dfcf4fa5-3952-4914-a9db-013f7be543b2","Type":"ContainerStarted","Data":"c7f8502c95eccb9c2b06e8b431eeeea2c96c7b5af53e1845ff61666387ebca10"} Jul 30 02:30:56.755816 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:56.755773 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t7kcf" podStartSLOduration=2.394772869 podStartE2EDuration="3.755760121s" totalImagesPullingTime="1.289002282s" totalInitContainerRuntime="71.98497ms" isStatefulPod=true podCreationTimestamp="2026-07-30 02:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:30:56.755040646 +0000 UTC m=+304.433844653" watchObservedRunningTime="2026-07-30 02:30:56.755760121 +0000 UTC m=+304.434564115" Jul 30 02:30:56.816682 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:56.816640 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:56.926461 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:56.926377 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:57.026892 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:57.026851 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:57.126537 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:57.126513 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:57.227801 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:57.227727 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:57.340438 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:57.340415 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:57.440192 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:57.440167 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:57.540967 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:57.540934 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:57.563156 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:57.563133 2566 patch_prober.go:28] interesting pod/image-registry-6c7765b4fc-p4gvx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 30 02:30:57.563257 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:57.563175 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" podUID="ca64202a-9a1b-42d4-b980-cabdcddc4976" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 02:30:57.671900 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:57.671861 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c7765b4fc-p4gvx"] Jul 30 02:30:57.676009 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:57.675984 2566 patch_prober.go:28] interesting pod/image-registry-6c7765b4fc-p4gvx container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 30 02:30:57.676134 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:57.676036 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" podUID="ca64202a-9a1b-42d4-b980-cabdcddc4976" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 02:30:57.745161 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:57.745128 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:58.041323 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.041296 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-76d86795b9-jm949"] Jul 30 02:30:58.044440 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.044425 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.046964 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.046937 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pdbtw\"" Jul 30 02:30:58.046964 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.046958 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 30 02:30:58.047191 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.047037 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 30 02:30:58.047191 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.047051 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 30 02:30:58.047300 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.047240 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 30 02:30:58.047696 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.047678 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2abb95fj613lh\"" Jul 30 02:30:58.053907 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.053888 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76d86795b9-jm949"] Jul 30 02:30:58.070141 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.070116 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:58.175875 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.175851 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:58.231253 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.231228 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bcdf28eb-92fa-4329-8ef3-141f5edba294-client-ca-bundle\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.231335 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.231279 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcdf28eb-92fa-4329-8ef3-141f5edba294-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.231401 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.231370 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bcdf28eb-92fa-4329-8ef3-141f5edba294-metrics-server-audit-profiles\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.231460 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.231411 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jlt6\" (UniqueName: \"kubernetes.io/projected/bcdf28eb-92fa-4329-8ef3-141f5edba294-kube-api-access-8jlt6\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.231512 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.231456 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bcdf28eb-92fa-4329-8ef3-141f5edba294-secret-metrics-server-client-certs\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.231512 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.231482 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bcdf28eb-92fa-4329-8ef3-141f5edba294-secret-metrics-server-tls\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.231603 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.231520 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bcdf28eb-92fa-4329-8ef3-141f5edba294-audit-log\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.232120 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.232098 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:58.332544 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.332489 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bcdf28eb-92fa-4329-8ef3-141f5edba294-secret-metrics-server-client-certs\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.332544 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.332520 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bcdf28eb-92fa-4329-8ef3-141f5edba294-secret-metrics-server-tls\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.332656 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.332549 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bcdf28eb-92fa-4329-8ef3-141f5edba294-audit-log\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.332656 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.332594 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bcdf28eb-92fa-4329-8ef3-141f5edba294-client-ca-bundle\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.332656 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.332627 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcdf28eb-92fa-4329-8ef3-141f5edba294-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.332827 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.332687 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bcdf28eb-92fa-4329-8ef3-141f5edba294-metrics-server-audit-profiles\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.332827 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.332722 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jlt6\" (UniqueName: \"kubernetes.io/projected/bcdf28eb-92fa-4329-8ef3-141f5edba294-kube-api-access-8jlt6\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.332906 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.332893 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bcdf28eb-92fa-4329-8ef3-141f5edba294-audit-log\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.333335 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.333313 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcdf28eb-92fa-4329-8ef3-141f5edba294-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.333808 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.333784 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bcdf28eb-92fa-4329-8ef3-141f5edba294-metrics-server-audit-profiles\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.335062 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.335044 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bcdf28eb-92fa-4329-8ef3-141f5edba294-secret-metrics-server-client-certs\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.335270 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.335251 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bcdf28eb-92fa-4329-8ef3-141f5edba294-client-ca-bundle\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.335307 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.335261 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bcdf28eb-92fa-4329-8ef3-141f5edba294-secret-metrics-server-tls\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.339793 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.339772 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jlt6\" (UniqueName: \"kubernetes.io/projected/bcdf28eb-92fa-4329-8ef3-141f5edba294-kube-api-access-8jlt6\") pod \"metrics-server-76d86795b9-jm949\" (UID: \"bcdf28eb-92fa-4329-8ef3-141f5edba294\") " pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.353673 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.353649 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:30:58.455883 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.455852 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:58.484310 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.484285 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76d86795b9-jm949"] Jul 30 02:30:58.486825 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:30:58.486800 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcdf28eb_92fa_4329_8ef3_141f5edba294.slice/crio-cb47d29e3f9044ada1ce8b3ac69972602a3aefc70c2e6e086729b5c35c1c8b75 WatchSource:0}: Error finding container cb47d29e3f9044ada1ce8b3ac69972602a3aefc70c2e6e086729b5c35c1c8b75: Status 404 returned error can't find the container with id cb47d29e3f9044ada1ce8b3ac69972602a3aefc70c2e6e086729b5c35c1c8b75 Jul 30 02:30:58.555875 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.555843 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:58.745464 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.745377 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:58.747617 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.747595 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76d86795b9-jm949" event={"ID":"bcdf28eb-92fa-4329-8ef3-141f5edba294","Type":"ContainerStarted","Data":"cb47d29e3f9044ada1ce8b3ac69972602a3aefc70c2e6e086729b5c35c1c8b75"} Jul 30 02:30:58.795075 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:58.794794 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:59.883465 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.883430 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 30 02:30:59.887866 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.887839 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.889189 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.888841 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:30:59.890925 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.890758 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 30 02:30:59.890925 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.890758 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 30 02:30:59.890925 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.890816 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 30 02:30:59.890925 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.890868 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 30 02:30:59.890925 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.890914 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 30 02:30:59.891604 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.891582 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4v8sq\"" Jul 30 02:30:59.891829 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.891811 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4gc2ba6ecl8c\"" Jul 30 02:30:59.891940 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.891837 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 30 02:30:59.892010 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.891950 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 30 02:30:59.893718 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.893519 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 30 02:30:59.894758 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.894202 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 30 02:30:59.894758 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.894438 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 30 02:30:59.894758 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.894546 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 30 02:30:59.898274 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.895055 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 30 02:30:59.898274 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.895097 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 30 02:30:59.903886 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.902372 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 30 02:30:59.904682 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.904643 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 30 02:30:59.947068 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947036 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947162 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947074 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c73fb42-d32d-4e24-a493-9f77a39b3591-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947162 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947103 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsp22\" (UniqueName: \"kubernetes.io/projected/5c73fb42-d32d-4e24-a493-9f77a39b3591-kube-api-access-fsp22\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947162 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947127 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c73fb42-d32d-4e24-a493-9f77a39b3591-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947162 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947151 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947355 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947179 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947355 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947204 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947355 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947258 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947355 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947307 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-config\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947355 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947331 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947355 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947353 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947632 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947378 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947632 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947406 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947632 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947431 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947632 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947454 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947632 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947482 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947632 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947507 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947632 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947558 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.947632 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947597 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:30:59.948007 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:30:59.947643 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048189 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048144 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048189 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048191 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048422 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048217 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048422 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048422 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048293 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048422 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048326 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048422 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048367 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048422 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048395 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048735 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048428 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c73fb42-d32d-4e24-a493-9f77a39b3591-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048735 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048454 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsp22\" (UniqueName: \"kubernetes.io/projected/5c73fb42-d32d-4e24-a493-9f77a39b3591-kube-api-access-fsp22\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048735 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048479 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c73fb42-d32d-4e24-a493-9f77a39b3591-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048735 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048506 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048735 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048534 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048735 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048559 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048735 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048591 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048735 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048628 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-config\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048735 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048652 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048735 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048691 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.048735 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048718 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.049263 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.049263 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.048947 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.049263 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.049007 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.049413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.049289 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.049413 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.049371 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.049513 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.049415 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.049923 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.049894 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.052813 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.052198 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.052813 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.052756 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.054731 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.054705 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c73fb42-d32d-4e24-a493-9f77a39b3591-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.055782 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.055689 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.055782 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.055697 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.055782 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.055727 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.056411 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.056177 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.056411 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.056315 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.056605 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.056584 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c73fb42-d32d-4e24-a493-9f77a39b3591-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.056761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.056648 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.057351 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.057321 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.057438 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.057406 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-config\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.057736 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.057719 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.062217 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.062176 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsp22\" (UniqueName: \"kubernetes.io/projected/5c73fb42-d32d-4e24-a493-9f77a39b3591-kube-api-access-fsp22\") pod \"prometheus-k8s-0\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.206358 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.206272 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:00.365383 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.365356 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 30 02:31:00.367851 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:31:00.367821 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c73fb42_d32d_4e24_a493_9f77a39b3591.slice/crio-f9c3d0430bcbd187355f54ed430af8ceaa659fbc6af48a3fc84f7cdca5477aa0 WatchSource:0}: Error finding container f9c3d0430bcbd187355f54ed430af8ceaa659fbc6af48a3fc84f7cdca5477aa0: Status 404 returned error can't find the container with id f9c3d0430bcbd187355f54ed430af8ceaa659fbc6af48a3fc84f7cdca5477aa0 Jul 30 02:31:00.756620 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.756582 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerStarted","Data":"f9c3d0430bcbd187355f54ed430af8ceaa659fbc6af48a3fc84f7cdca5477aa0"} Jul 30 02:31:00.757975 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.757944 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76d86795b9-jm949" event={"ID":"bcdf28eb-92fa-4329-8ef3-141f5edba294","Type":"ContainerStarted","Data":"7aefd00cab232c783a07817e6bceca4c1be71a8b15af9e6a3daba470a27e3ab1"} Jul 30 02:31:00.774933 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.774871 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-76d86795b9-jm949" podStartSLOduration=1.329599986 podStartE2EDuration="2.774853179s" totalImagesPullingTime="1.445253193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:31:00.773426588 +0000 UTC m=+308.452230584" watchObservedRunningTime="2026-07-30 02:31:00.774853179 +0000 UTC m=+308.453657177" Jul 30 02:31:00.846945 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:00.846916 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:31:01.763851 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:01.763755 2566 generic.go:356] "Generic (PLEG): container finished" podID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerID="2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831" exitCode=0 Jul 30 02:31:01.764258 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:01.763851 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerDied","Data":"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831"} Jul 30 02:31:02.730455 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:02.730419 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:02.831316 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:02.831274 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:04.755440 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:04.755400 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:04.855691 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:04.855636 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:05.784118 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:05.784080 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerStarted","Data":"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba"} Jul 30 02:31:05.784450 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:05.784178 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerStarted","Data":"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13"} Jul 30 02:31:07.678150 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:07.678121 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:31:07.796233 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:07.796198 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerStarted","Data":"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da"} Jul 30 02:31:07.796409 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:07.796257 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerStarted","Data":"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888"} Jul 30 02:31:07.796409 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:07.796284 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerStarted","Data":"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1"} Jul 30 02:31:07.796409 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:07.796307 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerStarted","Data":"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056"} Jul 30 02:31:07.822469 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:07.822401 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.457093454 podStartE2EDuration="8.822382879s" totalImagesPullingTime="6.345350351s" totalInitContainerRuntime="19.939074ms" isStatefulPod=false podCreationTimestamp="2026-07-30 02:30:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:31:07.820890584 +0000 UTC m=+315.499694579" watchObservedRunningTime="2026-07-30 02:31:07.822382879 +0000 UTC m=+315.501186875" Jul 30 02:31:10.206416 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:10.206382 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:31:10.969627 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:10.969583 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:11.021079 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:11.021039 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:11.811867 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:11.811835 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:11.866943 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:11.866913 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:12.133859 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:12.133783 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:12.188416 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:12.188389 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:14.754864 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:14.754829 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:14.804778 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:14.804748 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:18.354605 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:18.354572 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:31:18.354988 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:18.354630 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:31:20.204170 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:20.204136 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:20.320135 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:20.320099 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:20.816945 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:20.816916 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:20.915682 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:20.915636 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:21.174083 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:21.173995 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:21.229883 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:21.229855 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:21.489723 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:21.489634 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:22.174457 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:22.174408 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:22.239332 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:22.239294 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:22.690261 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:22.690224 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" podUID="ca64202a-9a1b-42d4-b980-cabdcddc4976" containerName="registry" containerID="cri-o://09548b83edc4ef6ba9bd09855af6e4a27d56226b90f92125540d5e4bcfa66c38" gracePeriod=30 Jul 30 02:31:22.855846 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:22.855812 2566 generic.go:356] "Generic (PLEG): container finished" podID="ca64202a-9a1b-42d4-b980-cabdcddc4976" containerID="09548b83edc4ef6ba9bd09855af6e4a27d56226b90f92125540d5e4bcfa66c38" exitCode=0 Jul 30 02:31:22.856299 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:22.856275 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" event={"ID":"ca64202a-9a1b-42d4-b980-cabdcddc4976","Type":"ContainerDied","Data":"09548b83edc4ef6ba9bd09855af6e4a27d56226b90f92125540d5e4bcfa66c38"} Jul 30 02:31:22.947063 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:22.947014 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:31:23.026247 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.026217 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ca64202a-9a1b-42d4-b980-cabdcddc4976-ca-trust-extracted\") pod \"ca64202a-9a1b-42d4-b980-cabdcddc4976\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " Jul 30 02:31:23.026362 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.026281 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-bound-sa-token\") pod \"ca64202a-9a1b-42d4-b980-cabdcddc4976\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " Jul 30 02:31:23.026362 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.026310 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") pod \"ca64202a-9a1b-42d4-b980-cabdcddc4976\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " Jul 30 02:31:23.026362 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.026348 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ca64202a-9a1b-42d4-b980-cabdcddc4976-trusted-ca\") pod \"ca64202a-9a1b-42d4-b980-cabdcddc4976\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " Jul 30 02:31:23.026525 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.026470 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-certificates\") pod \"ca64202a-9a1b-42d4-b980-cabdcddc4976\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " Jul 30 02:31:23.026525 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.026511 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bq2xw\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-kube-api-access-bq2xw\") pod \"ca64202a-9a1b-42d4-b980-cabdcddc4976\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " Jul 30 02:31:23.026627 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.026550 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ca64202a-9a1b-42d4-b980-cabdcddc4976-image-registry-private-configuration\") pod \"ca64202a-9a1b-42d4-b980-cabdcddc4976\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " Jul 30 02:31:23.026627 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.026575 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ca64202a-9a1b-42d4-b980-cabdcddc4976-installation-pull-secrets\") pod \"ca64202a-9a1b-42d4-b980-cabdcddc4976\" (UID: \"ca64202a-9a1b-42d4-b980-cabdcddc4976\") " Jul 30 02:31:23.026808 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.026781 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca64202a-9a1b-42d4-b980-cabdcddc4976-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ca64202a-9a1b-42d4-b980-cabdcddc4976" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 02:31:23.026940 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.026907 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ca64202a-9a1b-42d4-b980-cabdcddc4976-trusted-ca\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:31:23.026940 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.026917 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ca64202a-9a1b-42d4-b980-cabdcddc4976" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 02:31:23.028985 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.028949 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca64202a-9a1b-42d4-b980-cabdcddc4976-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ca64202a-9a1b-42d4-b980-cabdcddc4976" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 02:31:23.028985 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.028973 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ca64202a-9a1b-42d4-b980-cabdcddc4976" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 02:31:23.029179 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.029079 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-kube-api-access-bq2xw" (OuterVolumeSpecName: "kube-api-access-bq2xw") pod "ca64202a-9a1b-42d4-b980-cabdcddc4976" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976"). InnerVolumeSpecName "kube-api-access-bq2xw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 02:31:23.029305 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.029280 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ca64202a-9a1b-42d4-b980-cabdcddc4976" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 02:31:23.029341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.029327 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca64202a-9a1b-42d4-b980-cabdcddc4976-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ca64202a-9a1b-42d4-b980-cabdcddc4976" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 02:31:23.037031 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.037005 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca64202a-9a1b-42d4-b980-cabdcddc4976-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ca64202a-9a1b-42d4-b980-cabdcddc4976" (UID: "ca64202a-9a1b-42d4-b980-cabdcddc4976"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 02:31:23.127580 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.127557 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ca64202a-9a1b-42d4-b980-cabdcddc4976-ca-trust-extracted\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:31:23.127580 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.127578 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-bound-sa-token\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:31:23.127739 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.127588 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-tls\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:31:23.127739 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.127595 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ca64202a-9a1b-42d4-b980-cabdcddc4976-registry-certificates\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:31:23.127739 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.127603 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bq2xw\" (UniqueName: \"kubernetes.io/projected/ca64202a-9a1b-42d4-b980-cabdcddc4976-kube-api-access-bq2xw\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:31:23.127739 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.127613 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ca64202a-9a1b-42d4-b980-cabdcddc4976-image-registry-private-configuration\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:31:23.127739 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.127622 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ca64202a-9a1b-42d4-b980-cabdcddc4976-installation-pull-secrets\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:31:23.860670 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.860639 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" event={"ID":"ca64202a-9a1b-42d4-b980-cabdcddc4976","Type":"ContainerDied","Data":"5be63e733fae273616bfd2ef95912b0e87fded6e33e87397d7ee91c40f0aafeb"} Jul 30 02:31:23.860670 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.860679 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c7765b4fc-p4gvx" Jul 30 02:31:23.861197 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.860709 2566 scope.go:117] "RemoveContainer" containerID="09548b83edc4ef6ba9bd09855af6e4a27d56226b90f92125540d5e4bcfa66c38" Jul 30 02:31:23.884331 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.884306 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c7765b4fc-p4gvx"] Jul 30 02:31:23.893179 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.893159 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6c7765b4fc-p4gvx"] Jul 30 02:31:23.969846 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:23.969817 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:24.024786 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:24.024752 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:24.852066 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:24.852034 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca64202a-9a1b-42d4-b980-cabdcddc4976" path="/var/lib/kubelet/pods/ca64202a-9a1b-42d4-b980-cabdcddc4976/volumes" Jul 30 02:31:25.162861 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:25.162776 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:25.213833 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:25.213804 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:34.949159 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:34.949122 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:35.178193 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:35.178154 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:35.226703 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:35.226607 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:31:38.359968 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:38.359933 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:31:38.363787 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:38.363764 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-76d86795b9-jm949" Jul 30 02:31:43.522829 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:31:43.522796 2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda42dc3f7_c8d2_4e06_a0ea_69b8087a7920.slice/crio-conmon-24193e178338f3187179f74ec64b35b9b4e2199a01ba4d8327288acdeea1a3ba.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda42dc3f7_c8d2_4e06_a0ea_69b8087a7920.slice/crio-24193e178338f3187179f74ec64b35b9b4e2199a01ba4d8327288acdeea1a3ba.scope\": RecentStats: unable to find data in memory cache]" Jul 30 02:31:43.934050 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:43.934014 2566 generic.go:356] "Generic (PLEG): container finished" podID="a42dc3f7-c8d2-4e06-a0ea-69b8087a7920" containerID="24193e178338f3187179f74ec64b35b9b4e2199a01ba4d8327288acdeea1a3ba" exitCode=0 Jul 30 02:31:43.934210 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:43.934086 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" event={"ID":"a42dc3f7-c8d2-4e06-a0ea-69b8087a7920","Type":"ContainerDied","Data":"24193e178338f3187179f74ec64b35b9b4e2199a01ba4d8327288acdeea1a3ba"} Jul 30 02:31:43.934430 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:43.934414 2566 scope.go:117] "RemoveContainer" containerID="24193e178338f3187179f74ec64b35b9b4e2199a01ba4d8327288acdeea1a3ba" Jul 30 02:31:44.939896 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:31:44.939823 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vbkt5" event={"ID":"a42dc3f7-c8d2-4e06-a0ea-69b8087a7920","Type":"ContainerStarted","Data":"6c6bf5da7c0e61c1db6662ca7c6138abe247310e1716bd8ec319c6e245cab87e"} Jul 30 02:32:00.206525 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:00.206493 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:00.227366 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:00.227343 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:01.012215 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:01.012190 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:06.664838 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:06.664788 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:32:06.667069 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:06.667036 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db23d664-ec37-4206-b1f3-ac52ecec65dc-metrics-certs\") pod \"network-metrics-daemon-rx95s\" (UID: \"db23d664-ec37-4206-b1f3-ac52ecec65dc\") " pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:32:06.849853 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:06.849826 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vvjjn\"" Jul 30 02:32:06.857993 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:06.857974 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx95s" Jul 30 02:32:06.977910 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:06.977842 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rx95s"] Jul 30 02:32:06.980554 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:32:06.980532 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb23d664_ec37_4206_b1f3_ac52ecec65dc.slice/crio-d2ab16ff6c22caf4c00977fc37126965b0f11b969c8fe38963da2a874e74f30c WatchSource:0}: Error finding container d2ab16ff6c22caf4c00977fc37126965b0f11b969c8fe38963da2a874e74f30c: Status 404 returned error can't find the container with id d2ab16ff6c22caf4c00977fc37126965b0f11b969c8fe38963da2a874e74f30c Jul 30 02:32:07.018553 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:07.018525 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rx95s" event={"ID":"db23d664-ec37-4206-b1f3-ac52ecec65dc","Type":"ContainerStarted","Data":"d2ab16ff6c22caf4c00977fc37126965b0f11b969c8fe38963da2a874e74f30c"} Jul 30 02:32:09.029165 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:09.029129 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rx95s" event={"ID":"db23d664-ec37-4206-b1f3-ac52ecec65dc","Type":"ContainerStarted","Data":"13f4c7a05676c9756412048151283e63a10a8c85c3ee4fd52f7bef1f708cee6c"} Jul 30 02:32:09.029557 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:09.029194 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rx95s" event={"ID":"db23d664-ec37-4206-b1f3-ac52ecec65dc","Type":"ContainerStarted","Data":"00e11556f8ed5e8332c5f12d4b27d239af09fe35c50e17b526e04f31b72285f3"} Jul 30 02:32:09.045808 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:09.045764 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rx95s" podStartSLOduration=374.851499375 podStartE2EDuration="6m16.045749034s" totalImagesPullingTime="1.194249659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:32:09.044412224 +0000 UTC m=+376.723216220" watchObservedRunningTime="2026-07-30 02:32:09.045749034 +0000 UTC m=+376.724553028" Jul 30 02:32:09.176182 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:09.176156 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:09.236026 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:09.235992 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:09.757845 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:09.757767 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:09.812644 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:09.812618 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:18.256837 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.256804 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 30 02:32:18.257252 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.257169 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="prometheus" containerID="cri-o://8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13" gracePeriod=600 Jul 30 02:32:18.258559 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.257361 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="kube-rbac-proxy-thanos" containerID="cri-o://1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da" gracePeriod=600 Jul 30 02:32:18.258559 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.257416 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="thanos-sidecar" containerID="cri-o://ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056" gracePeriod=600 Jul 30 02:32:18.258559 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.257560 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="kube-rbac-proxy" containerID="cri-o://cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888" gracePeriod=600 Jul 30 02:32:18.258559 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.257641 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="kube-rbac-proxy-web" containerID="cri-o://1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1" gracePeriod=600 Jul 30 02:32:18.258559 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.257749 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="config-reloader" containerID="cri-o://248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba" gracePeriod=600 Jul 30 02:32:18.514524 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.514463 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:18.550539 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550514 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c73fb42-d32d-4e24-a493-9f77a39b3591-config-out\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.550539 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550542 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-db\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.550847 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550566 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsp22\" (UniqueName: \"kubernetes.io/projected/5c73fb42-d32d-4e24-a493-9f77a39b3591-kube-api-access-fsp22\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.550847 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550607 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.550847 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550635 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-config\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.550847 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550680 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-1\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.550847 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550705 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-0\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.550847 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550741 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-grpc-tls\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.550847 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550770 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.550847 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550805 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-kube-rbac-proxy\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.550847 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550844 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-metrics-client-certs\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.551356 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550876 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-kubelet-serving-ca-bundle\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.551356 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550911 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-serving-certs-ca-bundle\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.551356 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550935 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-tls\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.551356 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.550980 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-thanos-prometheus-http-client-file\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.551356 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.551008 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-metrics-client-ca\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.551356 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.551022 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 02:32:18.551356 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.551046 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-web-config\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.551356 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.551078 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c73fb42-d32d-4e24-a493-9f77a39b3591-tls-assets\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.551356 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.551105 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-2\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.551356 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.551129 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-trusted-ca-bundle\") pod \"5c73fb42-d32d-4e24-a493-9f77a39b3591\" (UID: \"5c73fb42-d32d-4e24-a493-9f77a39b3591\") " Jul 30 02:32:18.551356 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.551362 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.551996 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.551657 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 02:32:18.551996 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.551699 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 02:32:18.553110 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.552464 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 02:32:18.553110 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.552836 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 02:32:18.553110 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.552833 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 02:32:18.553333 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.553117 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 02:32:18.553387 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.553349 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 02:32:18.555631 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.555459 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c73fb42-d32d-4e24-a493-9f77a39b3591-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 02:32:18.555631 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.555529 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c73fb42-d32d-4e24-a493-9f77a39b3591-kube-api-access-fsp22" (OuterVolumeSpecName: "kube-api-access-fsp22") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "kube-api-access-fsp22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 02:32:18.555631 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.555608 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-config" (OuterVolumeSpecName: "config") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 02:32:18.557173 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.557138 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 02:32:18.557795 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.557640 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 02:32:18.557795 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.557765 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 02:32:18.558001 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.557979 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c73fb42-d32d-4e24-a493-9f77a39b3591-config-out" (OuterVolumeSpecName: "config-out") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 02:32:18.558331 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.558302 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 02:32:18.558568 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.558547 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 02:32:18.559215 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.559180 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 02:32:18.559305 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.559281 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 02:32:18.572679 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.572633 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-web-config" (OuterVolumeSpecName: "web-config") pod "5c73fb42-d32d-4e24-a493-9f77a39b3591" (UID: "5c73fb42-d32d-4e24-a493-9f77a39b3591"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 02:32:18.652043 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652005 2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c73fb42-d32d-4e24-a493-9f77a39b3591-config-out\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652043 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652042 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-db\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652058 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsp22\" (UniqueName: \"kubernetes.io/projected/5c73fb42-d32d-4e24-a493-9f77a39b3591-kube-api-access-fsp22\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652074 2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652086 2566 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-config\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652098 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652110 2566 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-grpc-tls\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652123 2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652135 2566 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-kube-rbac-proxy\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652147 2566 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-metrics-client-certs\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652161 2566 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652173 2566 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652187 2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652201 2566 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652214 2566 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-configmap-metrics-client-ca\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652617 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652230 2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c73fb42-d32d-4e24-a493-9f77a39b3591-web-config\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652617 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652243 2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c73fb42-d32d-4e24-a493-9f77a39b3591-tls-assets\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652617 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652256 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:18.652617 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:18.652269 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c73fb42-d32d-4e24-a493-9f77a39b3591-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:32:19.071761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.071731 2566 generic.go:356] "Generic (PLEG): container finished" podID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerID="1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da" exitCode=0 Jul 30 02:32:19.071761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.071756 2566 generic.go:356] "Generic (PLEG): container finished" podID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerID="cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888" exitCode=0 Jul 30 02:32:19.071761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.071762 2566 generic.go:356] "Generic (PLEG): container finished" podID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerID="1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1" exitCode=0 Jul 30 02:32:19.071761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.071768 2566 generic.go:356] "Generic (PLEG): container finished" podID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerID="ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056" exitCode=0 Jul 30 02:32:19.072035 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.071774 2566 generic.go:356] "Generic (PLEG): container finished" podID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerID="248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba" exitCode=0 Jul 30 02:32:19.072035 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.071779 2566 generic.go:356] "Generic (PLEG): container finished" podID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerID="8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13" exitCode=0 Jul 30 02:32:19.072035 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.071808 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.072035 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.071807 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerDied","Data":"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da"} Jul 30 02:32:19.072035 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.071875 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerDied","Data":"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888"} Jul 30 02:32:19.072035 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.071903 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerDied","Data":"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1"} Jul 30 02:32:19.072035 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.071933 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerDied","Data":"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056"} Jul 30 02:32:19.072035 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.071958 2566 scope.go:117] "RemoveContainer" containerID="1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da" Jul 30 02:32:19.072035 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.071968 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerDied","Data":"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba"} Jul 30 02:32:19.072440 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.072071 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerDied","Data":"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13"} Jul 30 02:32:19.072440 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.072095 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c73fb42-d32d-4e24-a493-9f77a39b3591","Type":"ContainerDied","Data":"f9c3d0430bcbd187355f54ed430af8ceaa659fbc6af48a3fc84f7cdca5477aa0"} Jul 30 02:32:19.079558 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.079541 2566 scope.go:117] "RemoveContainer" containerID="cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888" Jul 30 02:32:19.085898 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.085882 2566 scope.go:117] "RemoveContainer" containerID="1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1" Jul 30 02:32:19.092410 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.092295 2566 scope.go:117] "RemoveContainer" containerID="ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056" Jul 30 02:32:19.094006 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.093984 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 30 02:32:19.099159 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.099136 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 30 02:32:19.099314 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.099269 2566 scope.go:117] "RemoveContainer" containerID="248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba" Jul 30 02:32:19.105514 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.105497 2566 scope.go:117] "RemoveContainer" containerID="8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13" Jul 30 02:32:19.112320 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.112305 2566 scope.go:117] "RemoveContainer" containerID="2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831" Jul 30 02:32:19.118419 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.118406 2566 scope.go:117] "RemoveContainer" containerID="1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da" Jul 30 02:32:19.118653 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:32:19.118636 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da\": container with ID starting with 1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da not found: ID does not exist" containerID="1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da" Jul 30 02:32:19.118713 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.118674 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da"} err="failed to get container status \"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da\": rpc error: code = NotFound desc = could not find container \"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da\": container with ID starting with 1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da not found: ID does not exist" Jul 30 02:32:19.118761 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.118713 2566 scope.go:117] "RemoveContainer" containerID="cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888" Jul 30 02:32:19.118912 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:32:19.118897 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888\": container with ID starting with cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888 not found: ID does not exist" containerID="cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888" Jul 30 02:32:19.119014 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.118918 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888"} err="failed to get container status \"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888\": rpc error: code = NotFound desc = could not find container \"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888\": container with ID starting with cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888 not found: ID does not exist" Jul 30 02:32:19.119014 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.118936 2566 scope.go:117] "RemoveContainer" containerID="1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1" Jul 30 02:32:19.119192 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:32:19.119176 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1\": container with ID starting with 1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1 not found: ID does not exist" containerID="1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1" Jul 30 02:32:19.119249 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.119197 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1"} err="failed to get container status \"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1\": rpc error: code = NotFound desc = could not find container \"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1\": container with ID starting with 1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1 not found: ID does not exist" Jul 30 02:32:19.119249 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.119212 2566 scope.go:117] "RemoveContainer" containerID="ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056" Jul 30 02:32:19.119481 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:32:19.119460 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056\": container with ID starting with ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056 not found: ID does not exist" containerID="ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056" Jul 30 02:32:19.119549 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.119489 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056"} err="failed to get container status \"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056\": rpc error: code = NotFound desc = could not find container \"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056\": container with ID starting with ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056 not found: ID does not exist" Jul 30 02:32:19.119549 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.119512 2566 scope.go:117] "RemoveContainer" containerID="248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba" Jul 30 02:32:19.119805 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:32:19.119789 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba\": container with ID starting with 248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba not found: ID does not exist" containerID="248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba" Jul 30 02:32:19.119877 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.119811 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba"} err="failed to get container status \"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba\": rpc error: code = NotFound desc = could not find container \"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba\": container with ID starting with 248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba not found: ID does not exist" Jul 30 02:32:19.119877 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.119831 2566 scope.go:117] "RemoveContainer" containerID="8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13" Jul 30 02:32:19.120061 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:32:19.120045 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13\": container with ID starting with 8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13 not found: ID does not exist" containerID="8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13" Jul 30 02:32:19.120127 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.120068 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13"} err="failed to get container status \"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13\": rpc error: code = NotFound desc = could not find container \"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13\": container with ID starting with 8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13 not found: ID does not exist" Jul 30 02:32:19.120127 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.120086 2566 scope.go:117] "RemoveContainer" containerID="2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831" Jul 30 02:32:19.120313 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:32:19.120297 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831\": container with ID starting with 2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831 not found: ID does not exist" containerID="2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831" Jul 30 02:32:19.120349 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.120327 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831"} err="failed to get container status \"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831\": rpc error: code = NotFound desc = could not find container \"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831\": container with ID starting with 2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831 not found: ID does not exist" Jul 30 02:32:19.120349 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.120342 2566 scope.go:117] "RemoveContainer" containerID="1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da" Jul 30 02:32:19.120553 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.120539 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da"} err="failed to get container status \"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da\": rpc error: code = NotFound desc = could not find container \"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da\": container with ID starting with 1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da not found: ID does not exist" Jul 30 02:32:19.120553 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.120553 2566 scope.go:117] "RemoveContainer" containerID="cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888" Jul 30 02:32:19.120762 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.120743 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888"} err="failed to get container status \"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888\": rpc error: code = NotFound desc = could not find container \"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888\": container with ID starting with cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888 not found: ID does not exist" Jul 30 02:32:19.120835 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.120763 2566 scope.go:117] "RemoveContainer" containerID="1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1" Jul 30 02:32:19.120990 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.120970 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1"} err="failed to get container status \"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1\": rpc error: code = NotFound desc = could not find container \"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1\": container with ID starting with 1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1 not found: ID does not exist" Jul 30 02:32:19.121050 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.120992 2566 scope.go:117] "RemoveContainer" containerID="ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056" Jul 30 02:32:19.121212 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.121193 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056"} err="failed to get container status \"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056\": rpc error: code = NotFound desc = could not find container \"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056\": container with ID starting with ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056 not found: ID does not exist" Jul 30 02:32:19.121248 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.121213 2566 scope.go:117] "RemoveContainer" containerID="248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba" Jul 30 02:32:19.121445 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.121426 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba"} err="failed to get container status \"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba\": rpc error: code = NotFound desc = could not find container \"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba\": container with ID starting with 248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba not found: ID does not exist" Jul 30 02:32:19.121511 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.121446 2566 scope.go:117] "RemoveContainer" containerID="8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13" Jul 30 02:32:19.121674 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.121646 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13"} err="failed to get container status \"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13\": rpc error: code = NotFound desc = could not find container \"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13\": container with ID starting with 8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13 not found: ID does not exist" Jul 30 02:32:19.121745 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.121729 2566 scope.go:117] "RemoveContainer" containerID="2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831" Jul 30 02:32:19.121960 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.121943 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831"} err="failed to get container status \"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831\": rpc error: code = NotFound desc = could not find container \"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831\": container with ID starting with 2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831 not found: ID does not exist" Jul 30 02:32:19.122003 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.121960 2566 scope.go:117] "RemoveContainer" containerID="1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da" Jul 30 02:32:19.122164 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.122147 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da"} err="failed to get container status \"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da\": rpc error: code = NotFound desc = could not find container \"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da\": container with ID starting with 1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da not found: ID does not exist" Jul 30 02:32:19.122229 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.122166 2566 scope.go:117] "RemoveContainer" containerID="cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888" Jul 30 02:32:19.122383 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.122365 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888"} err="failed to get container status \"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888\": rpc error: code = NotFound desc = could not find container \"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888\": container with ID starting with cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888 not found: ID does not exist" Jul 30 02:32:19.122430 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.122384 2566 scope.go:117] "RemoveContainer" containerID="1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1" Jul 30 02:32:19.122572 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.122555 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1"} err="failed to get container status \"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1\": rpc error: code = NotFound desc = could not find container \"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1\": container with ID starting with 1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1 not found: ID does not exist" Jul 30 02:32:19.122635 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.122573 2566 scope.go:117] "RemoveContainer" containerID="ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056" Jul 30 02:32:19.122769 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.122754 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056"} err="failed to get container status \"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056\": rpc error: code = NotFound desc = could not find container \"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056\": container with ID starting with ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056 not found: ID does not exist" Jul 30 02:32:19.122819 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.122770 2566 scope.go:117] "RemoveContainer" containerID="248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba" Jul 30 02:32:19.122950 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.122933 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba"} err="failed to get container status \"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba\": rpc error: code = NotFound desc = could not find container \"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba\": container with ID starting with 248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba not found: ID does not exist" Jul 30 02:32:19.123017 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.122951 2566 scope.go:117] "RemoveContainer" containerID="8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13" Jul 30 02:32:19.123143 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.123129 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13"} err="failed to get container status \"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13\": rpc error: code = NotFound desc = could not find container \"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13\": container with ID starting with 8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13 not found: ID does not exist" Jul 30 02:32:19.123189 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.123143 2566 scope.go:117] "RemoveContainer" containerID="2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831" Jul 30 02:32:19.123331 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.123313 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831"} err="failed to get container status \"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831\": rpc error: code = NotFound desc = could not find container \"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831\": container with ID starting with 2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831 not found: ID does not exist" Jul 30 02:32:19.123367 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.123331 2566 scope.go:117] "RemoveContainer" containerID="1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da" Jul 30 02:32:19.123536 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.123519 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da"} err="failed to get container status \"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da\": rpc error: code = NotFound desc = could not find container \"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da\": container with ID starting with 1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da not found: ID does not exist" Jul 30 02:32:19.123597 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.123538 2566 scope.go:117] "RemoveContainer" containerID="cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888" Jul 30 02:32:19.123737 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.123720 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888"} err="failed to get container status \"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888\": rpc error: code = NotFound desc = could not find container \"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888\": container with ID starting with cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888 not found: ID does not exist" Jul 30 02:32:19.123785 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.123737 2566 scope.go:117] "RemoveContainer" containerID="1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1" Jul 30 02:32:19.123918 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.123900 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1"} err="failed to get container status \"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1\": rpc error: code = NotFound desc = could not find container \"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1\": container with ID starting with 1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1 not found: ID does not exist" Jul 30 02:32:19.123964 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.123919 2566 scope.go:117] "RemoveContainer" containerID="ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056" Jul 30 02:32:19.124109 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.124094 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056"} err="failed to get container status \"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056\": rpc error: code = NotFound desc = could not find container \"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056\": container with ID starting with ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056 not found: ID does not exist" Jul 30 02:32:19.124149 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.124108 2566 scope.go:117] "RemoveContainer" containerID="248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba" Jul 30 02:32:19.124263 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.124246 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba"} err="failed to get container status \"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba\": rpc error: code = NotFound desc = could not find container \"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba\": container with ID starting with 248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba not found: ID does not exist" Jul 30 02:32:19.124326 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.124264 2566 scope.go:117] "RemoveContainer" containerID="8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13" Jul 30 02:32:19.124492 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.124476 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13"} err="failed to get container status \"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13\": rpc error: code = NotFound desc = could not find container \"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13\": container with ID starting with 8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13 not found: ID does not exist" Jul 30 02:32:19.124536 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.124493 2566 scope.go:117] "RemoveContainer" containerID="2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831" Jul 30 02:32:19.124718 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.124698 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831"} err="failed to get container status \"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831\": rpc error: code = NotFound desc = could not find container \"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831\": container with ID starting with 2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831 not found: ID does not exist" Jul 30 02:32:19.124763 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.124720 2566 scope.go:117] "RemoveContainer" containerID="1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da" Jul 30 02:32:19.124922 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.124906 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da"} err="failed to get container status \"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da\": rpc error: code = NotFound desc = could not find container \"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da\": container with ID starting with 1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da not found: ID does not exist" Jul 30 02:32:19.124981 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.124924 2566 scope.go:117] "RemoveContainer" containerID="cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888" Jul 30 02:32:19.125090 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.125074 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888"} err="failed to get container status \"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888\": rpc error: code = NotFound desc = could not find container \"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888\": container with ID starting with cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888 not found: ID does not exist" Jul 30 02:32:19.125182 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.125090 2566 scope.go:117] "RemoveContainer" containerID="1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1" Jul 30 02:32:19.125287 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.125274 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1"} err="failed to get container status \"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1\": rpc error: code = NotFound desc = could not find container \"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1\": container with ID starting with 1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1 not found: ID does not exist" Jul 30 02:32:19.125332 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.125287 2566 scope.go:117] "RemoveContainer" containerID="ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056" Jul 30 02:32:19.125460 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.125444 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056"} err="failed to get container status \"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056\": rpc error: code = NotFound desc = could not find container \"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056\": container with ID starting with ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056 not found: ID does not exist" Jul 30 02:32:19.125523 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.125463 2566 scope.go:117] "RemoveContainer" containerID="248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba" Jul 30 02:32:19.125657 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.125642 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba"} err="failed to get container status \"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba\": rpc error: code = NotFound desc = could not find container \"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba\": container with ID starting with 248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba not found: ID does not exist" Jul 30 02:32:19.125721 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.125657 2566 scope.go:117] "RemoveContainer" containerID="8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13" Jul 30 02:32:19.125865 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.125845 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13"} err="failed to get container status \"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13\": rpc error: code = NotFound desc = could not find container \"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13\": container with ID starting with 8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13 not found: ID does not exist" Jul 30 02:32:19.125932 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.125865 2566 scope.go:117] "RemoveContainer" containerID="2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831" Jul 30 02:32:19.126069 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.126050 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831"} err="failed to get container status \"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831\": rpc error: code = NotFound desc = could not find container \"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831\": container with ID starting with 2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831 not found: ID does not exist" Jul 30 02:32:19.126178 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.126070 2566 scope.go:117] "RemoveContainer" containerID="1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da" Jul 30 02:32:19.126369 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.126337 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da"} err="failed to get container status \"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da\": rpc error: code = NotFound desc = could not find container \"1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da\": container with ID starting with 1e2997d034623d86f31c6b847d1396d7d313673b7d5ee8c74dd472da224f33da not found: ID does not exist" Jul 30 02:32:19.126369 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.126368 2566 scope.go:117] "RemoveContainer" containerID="cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888" Jul 30 02:32:19.126688 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.126647 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888"} err="failed to get container status \"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888\": rpc error: code = NotFound desc = could not find container \"cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888\": container with ID starting with cde0bd544556dcc1aee36f2ae1278b79749d248ae1362d4420ad7ec4d052a888 not found: ID does not exist" Jul 30 02:32:19.126688 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.126688 2566 scope.go:117] "RemoveContainer" containerID="1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1" Jul 30 02:32:19.126954 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.126932 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1"} err="failed to get container status \"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1\": rpc error: code = NotFound desc = could not find container \"1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1\": container with ID starting with 1697b02d1ef112aba2619625448a85b0306c1f02694b1f35819b65c8662c88b1 not found: ID does not exist" Jul 30 02:32:19.127049 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.126955 2566 scope.go:117] "RemoveContainer" containerID="ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056" Jul 30 02:32:19.127245 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.127218 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056"} err="failed to get container status \"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056\": rpc error: code = NotFound desc = could not find container \"ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056\": container with ID starting with ebe0e91b522370611c51c4563ee447090231fcf9a61481afc7d95641fc4ac056 not found: ID does not exist" Jul 30 02:32:19.127245 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.127241 2566 scope.go:117] "RemoveContainer" containerID="248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba" Jul 30 02:32:19.127531 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.127509 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba"} err="failed to get container status \"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba\": rpc error: code = NotFound desc = could not find container \"248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba\": container with ID starting with 248e1d2db8f0c593f5fc208f1775d8e791150c22ec0fdc5d293c1e89e22ad6ba not found: ID does not exist" Jul 30 02:32:19.127612 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.127533 2566 scope.go:117] "RemoveContainer" containerID="8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13" Jul 30 02:32:19.127810 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.127792 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13"} err="failed to get container status \"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13\": rpc error: code = NotFound desc = could not find container \"8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13\": container with ID starting with 8f2bb20857a78af73d2b0cc18b26d80261ab5f64d083e32f73756b7a8a43bf13 not found: ID does not exist" Jul 30 02:32:19.127902 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.127811 2566 scope.go:117] "RemoveContainer" containerID="2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831" Jul 30 02:32:19.128033 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128016 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831"} err="failed to get container status \"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831\": rpc error: code = NotFound desc = could not find container \"2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831\": container with ID starting with 2b86bad40f0ac74d6f948b360d8b85d7dc0754ade2d0b6ab41afa3a71f8f7831 not found: ID does not exist" Jul 30 02:32:19.128173 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128159 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 30 02:32:19.128411 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128401 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="config-reloader" Jul 30 02:32:19.128455 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128413 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="config-reloader" Jul 30 02:32:19.128455 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128423 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca64202a-9a1b-42d4-b980-cabdcddc4976" containerName="registry" Jul 30 02:32:19.128455 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128429 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca64202a-9a1b-42d4-b980-cabdcddc4976" containerName="registry" Jul 30 02:32:19.128455 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128435 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="kube-rbac-proxy-thanos" Jul 30 02:32:19.128455 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128440 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="kube-rbac-proxy-thanos" Jul 30 02:32:19.128455 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128456 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="kube-rbac-proxy" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128461 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="kube-rbac-proxy" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128471 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="init-config-reloader" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128476 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="init-config-reloader" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128486 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="kube-rbac-proxy-web" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128492 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="kube-rbac-proxy-web" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128499 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="thanos-sidecar" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128504 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="thanos-sidecar" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128517 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="prometheus" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128533 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="prometheus" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128575 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="thanos-sidecar" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128582 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="config-reloader" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128588 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca64202a-9a1b-42d4-b980-cabdcddc4976" containerName="registry" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128594 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="kube-rbac-proxy-web" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128600 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="prometheus" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128605 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="kube-rbac-proxy" Jul 30 02:32:19.128747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.128609 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" containerName="kube-rbac-proxy-thanos" Jul 30 02:32:19.133716 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.133700 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.136986 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.136819 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 30 02:32:19.137084 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.137034 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 30 02:32:19.137360 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.137219 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 30 02:32:19.137360 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.137292 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 30 02:32:19.137360 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.137311 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4v8sq\"" Jul 30 02:32:19.137540 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.137371 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4gc2ba6ecl8c\"" Jul 30 02:32:19.137688 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.137652 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 30 02:32:19.137790 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.137686 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 30 02:32:19.137844 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.137828 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 30 02:32:19.137994 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.137976 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 30 02:32:19.138168 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.138154 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 30 02:32:19.138431 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.138400 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 30 02:32:19.138431 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.138418 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 30 02:32:19.138645 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.138629 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 30 02:32:19.139964 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.139941 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 30 02:32:19.140147 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.140102 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:19.142675 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.142644 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 30 02:32:19.149242 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.149218 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 30 02:32:19.156115 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156091 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156208 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156126 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156208 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156146 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b808203-246f-4454-b334-f47f6908eedb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156208 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156162 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156324 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156234 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b808203-246f-4454-b334-f47f6908eedb-config-out\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156324 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156274 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156324 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156324 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156320 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzlnp\" (UniqueName: \"kubernetes.io/projected/7b808203-246f-4454-b334-f47f6908eedb-kube-api-access-mzlnp\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156460 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156392 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156460 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156418 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156460 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156444 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156571 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156474 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156571 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156490 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156644 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156565 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7b808203-246f-4454-b334-f47f6908eedb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156644 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156598 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156644 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156623 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-web-config\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156803 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156741 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156803 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156772 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-config\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156888 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156804 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.156888 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.156873 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.240159 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.240129 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:19.257732 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.257708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-config\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.257743 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.257773 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.257804 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.257832 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.257864 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b808203-246f-4454-b334-f47f6908eedb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.257887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.257911 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b808203-246f-4454-b334-f47f6908eedb-config-out\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.257934 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.257964 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.257990 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzlnp\" (UniqueName: \"kubernetes.io/projected/7b808203-246f-4454-b334-f47f6908eedb-kube-api-access-mzlnp\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.258025 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.258058 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.258082 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258116 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.258122 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258833 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.258144 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258833 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.258190 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7b808203-246f-4454-b334-f47f6908eedb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258833 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.258220 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258833 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.258243 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-web-config\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.258833 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.258301 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.259075 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.258865 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.259825 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.259465 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.260878 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.260787 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-config\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.260878 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.260838 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.261022 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.260946 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.261852 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.261701 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.261852 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.261788 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7b808203-246f-4454-b334-f47f6908eedb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.262000 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.261932 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b808203-246f-4454-b334-f47f6908eedb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.262067 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.261998 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.262827 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.262389 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.262827 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.262450 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.262827 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.262500 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.262827 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.262509 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.262827 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.262639 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.262827 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.262789 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b808203-246f-4454-b334-f47f6908eedb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.263174 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.263069 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b808203-246f-4454-b334-f47f6908eedb-config-out\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.263336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.263319 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.263534 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.263515 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.263804 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.263788 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b808203-246f-4454-b334-f47f6908eedb-web-config\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.270359 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.270334 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzlnp\" (UniqueName: \"kubernetes.io/projected/7b808203-246f-4454-b334-f47f6908eedb-kube-api-access-mzlnp\") pod \"prometheus-k8s-0\" (UID: \"7b808203-246f-4454-b334-f47f6908eedb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.445577 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.445506 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:19.582741 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:19.582719 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 30 02:32:19.584759 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:32:19.584731 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b808203_246f_4454_b334_f47f6908eedb.slice/crio-c4c46e6e49e4aad18af706823c491e20429cb77aba065d63362f37889bfbe156 WatchSource:0}: Error finding container c4c46e6e49e4aad18af706823c491e20429cb77aba065d63362f37889bfbe156: Status 404 returned error can't find the container with id c4c46e6e49e4aad18af706823c491e20429cb77aba065d63362f37889bfbe156 Jul 30 02:32:20.076690 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:20.076640 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:20.078115 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:20.078089 2566 generic.go:356] "Generic (PLEG): container finished" podID="7b808203-246f-4454-b334-f47f6908eedb" containerID="134d206bec3842c8c605db64ff17b3d82a17cd174648cdc2ba17e9e355a89557" exitCode=0 Jul 30 02:32:20.078239 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:20.078169 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b808203-246f-4454-b334-f47f6908eedb","Type":"ContainerDied","Data":"134d206bec3842c8c605db64ff17b3d82a17cd174648cdc2ba17e9e355a89557"} Jul 30 02:32:20.078291 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:20.078245 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b808203-246f-4454-b334-f47f6908eedb","Type":"ContainerStarted","Data":"c4c46e6e49e4aad18af706823c491e20429cb77aba065d63362f37889bfbe156"} Jul 30 02:32:20.170247 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:20.170220 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:20.851954 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:20.851924 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c73fb42-d32d-4e24-a493-9f77a39b3591" path="/var/lib/kubelet/pods/5c73fb42-d32d-4e24-a493-9f77a39b3591/volumes" Jul 30 02:32:21.085519 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:21.085491 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b808203-246f-4454-b334-f47f6908eedb","Type":"ContainerStarted","Data":"6144d7e2c0db35826b8420e04ec104d9664a352886654270328f9cf5b1592be1"} Jul 30 02:32:21.085649 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:21.085551 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b808203-246f-4454-b334-f47f6908eedb","Type":"ContainerStarted","Data":"3567a5ce15320e84ea0058304d29da082e4034baa63c1270342655cf459fb451"} Jul 30 02:32:21.085649 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:21.085578 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b808203-246f-4454-b334-f47f6908eedb","Type":"ContainerStarted","Data":"554018e0bf5de69c6a227b93fc1cc6f4aa8ff35d5f1a5b7c79d4299d3bd95d9c"} Jul 30 02:32:21.085649 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:21.085603 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b808203-246f-4454-b334-f47f6908eedb","Type":"ContainerStarted","Data":"47be4511edb67a1c3b86689aec6debb66db8fd3732df4d1828379329b0a02e31"} Jul 30 02:32:21.085649 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:21.085628 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b808203-246f-4454-b334-f47f6908eedb","Type":"ContainerStarted","Data":"b4b622ec6500fa1b91e69c95312a5c0084de1a11fcbb9759b144b4ca080383db"} Jul 30 02:32:21.085649 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:21.085649 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b808203-246f-4454-b334-f47f6908eedb","Type":"ContainerStarted","Data":"75fca7666cc9931d107c58bfd07d3738dbef710e75bc9c5dc92d9467ed4e6fa6"} Jul 30 02:32:21.114468 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:21.114367 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.092195767 podStartE2EDuration="2.114347571s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.151804ms" isStatefulPod=false podCreationTimestamp="2026-07-30 02:32:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:32:21.11214341 +0000 UTC m=+388.790947406" watchObservedRunningTime="2026-07-30 02:32:21.114347571 +0000 UTC m=+388.793151568" Jul 30 02:32:21.684224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:21.684191 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:21.735676 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:21.735633 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:24.446373 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:24.446330 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:32:31.246341 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:31.246300 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:31.304224 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:31.304202 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:31.387992 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:31.387963 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:31.447558 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:31.447535 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:32.053349 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:32.053316 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:32.109520 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:32.109482 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:35.188076 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:35.188039 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:35.288992 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:35.288953 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:42.174899 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:42.174863 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:32:42.227209 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:32:42.227172 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:14.637148 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:14.637064 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:14.732585 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:14.732544 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:19.445854 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:19.445815 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:33:19.461016 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:19.460994 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:33:20.326404 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:20.326375 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 30 02:33:23.672134 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:23.672098 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:23.726941 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:23.726902 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:24.438677 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:24.438623 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:41.344195 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:41.344156 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:42.149254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:42.149219 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:44.942419 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:44.942391 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:45.040814 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:45.040774 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:47.519626 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:47.519585 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:47.633473 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:47.633442 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:47.753697 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:47.753646 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:47.891847 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:47.891762 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:49.333923 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:49.333893 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:49.920559 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:49.920525 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:51.737336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:51.737297 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:51.793418 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:51.793388 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:54.900455 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:54.900421 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-s97g8"] Jul 30 02:33:54.903837 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:54.903819 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s97g8" Jul 30 02:33:54.906308 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:54.906290 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 30 02:33:54.907287 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:54.907271 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 30 02:33:54.907338 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:54.907300 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-vvrtb\"" Jul 30 02:33:54.913352 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:54.913329 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-s97g8"] Jul 30 02:33:55.000456 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:55.000416 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8880b97f-74dd-49fb-be19-645b1fc15999-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-s97g8\" (UID: \"8880b97f-74dd-49fb-be19-645b1fc15999\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s97g8" Jul 30 02:33:55.000621 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:55.000462 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9f89\" (UniqueName: \"kubernetes.io/projected/8880b97f-74dd-49fb-be19-645b1fc15999-kube-api-access-b9f89\") pod \"cert-manager-cainjector-559cdd8c87-s97g8\" (UID: \"8880b97f-74dd-49fb-be19-645b1fc15999\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s97g8" Jul 30 02:33:55.101883 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:55.101851 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9f89\" (UniqueName: \"kubernetes.io/projected/8880b97f-74dd-49fb-be19-645b1fc15999-kube-api-access-b9f89\") pod \"cert-manager-cainjector-559cdd8c87-s97g8\" (UID: \"8880b97f-74dd-49fb-be19-645b1fc15999\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s97g8" Jul 30 02:33:55.102051 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:55.101954 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8880b97f-74dd-49fb-be19-645b1fc15999-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-s97g8\" (UID: \"8880b97f-74dd-49fb-be19-645b1fc15999\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s97g8" Jul 30 02:33:55.110646 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:55.110620 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8880b97f-74dd-49fb-be19-645b1fc15999-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-s97g8\" (UID: \"8880b97f-74dd-49fb-be19-645b1fc15999\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s97g8" Jul 30 02:33:55.110789 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:55.110772 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9f89\" (UniqueName: \"kubernetes.io/projected/8880b97f-74dd-49fb-be19-645b1fc15999-kube-api-access-b9f89\") pod \"cert-manager-cainjector-559cdd8c87-s97g8\" (UID: \"8880b97f-74dd-49fb-be19-645b1fc15999\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-s97g8" Jul 30 02:33:55.222259 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:55.222175 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s97g8" Jul 30 02:33:55.352307 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:55.352282 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-s97g8"] Jul 30 02:33:55.354696 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:33:55.354650 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8880b97f_74dd_49fb_be19_645b1fc15999.slice/crio-4597a88d0dfaa5204631a18ed987e61b9ea7ee3fc076fd112faa20846933ef3b WatchSource:0}: Error finding container 4597a88d0dfaa5204631a18ed987e61b9ea7ee3fc076fd112faa20846933ef3b: Status 404 returned error can't find the container with id 4597a88d0dfaa5204631a18ed987e61b9ea7ee3fc076fd112faa20846933ef3b Jul 30 02:33:55.446852 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:55.446814 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s97g8" event={"ID":"8880b97f-74dd-49fb-be19-645b1fc15999","Type":"ContainerStarted","Data":"4597a88d0dfaa5204631a18ed987e61b9ea7ee3fc076fd112faa20846933ef3b"} Jul 30 02:33:58.725646 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:58.725608 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:59.220370 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:59.220319 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:33:59.321150 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:33:59.321110 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:00.468358 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:00.468272 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s97g8" event={"ID":"8880b97f-74dd-49fb-be19-645b1fc15999","Type":"ContainerStarted","Data":"6fc0b0067e1fa867bd98c859147559c670bc8862799eaf599acb5b714429aa07"} Jul 30 02:34:00.848515 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:00.848478 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:00.908186 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:00.908152 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:05.178155 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:05.178124 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:05.279787 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:05.279759 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:07.251207 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:07.251177 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:08.089747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:08.089693 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-s97g8" podStartSLOduration=9.373071634 podStartE2EDuration="14.089650534s" totalImagesPullingTime="4.7165789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:33:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:34:00.483757466 +0000 UTC m=+488.162561462" watchObservedRunningTime="2026-07-30 02:34:08.089650534 +0000 UTC m=+495.768454529" Jul 30 02:34:08.090717 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:08.090698 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fdpjd"] Jul 30 02:34:08.093759 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:08.093742 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fdpjd" Jul 30 02:34:08.096177 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:08.096156 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-nt4jg\"" Jul 30 02:34:08.102958 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:08.102932 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fdpjd"] Jul 30 02:34:08.211397 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:08.211359 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/450dcd0a-c95a-4c8c-b9cb-85b182475c72-bound-sa-token\") pod \"cert-manager-55b4d954d5-fdpjd\" (UID: \"450dcd0a-c95a-4c8c-b9cb-85b182475c72\") " pod="cert-manager/cert-manager-55b4d954d5-fdpjd" Jul 30 02:34:08.211560 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:08.211406 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c8c9\" (UniqueName: \"kubernetes.io/projected/450dcd0a-c95a-4c8c-b9cb-85b182475c72-kube-api-access-2c8c9\") pod \"cert-manager-55b4d954d5-fdpjd\" (UID: \"450dcd0a-c95a-4c8c-b9cb-85b182475c72\") " pod="cert-manager/cert-manager-55b4d954d5-fdpjd" Jul 30 02:34:08.311847 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:08.311807 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c8c9\" (UniqueName: \"kubernetes.io/projected/450dcd0a-c95a-4c8c-b9cb-85b182475c72-kube-api-access-2c8c9\") pod \"cert-manager-55b4d954d5-fdpjd\" (UID: \"450dcd0a-c95a-4c8c-b9cb-85b182475c72\") " pod="cert-manager/cert-manager-55b4d954d5-fdpjd" Jul 30 02:34:08.312301 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:08.311902 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/450dcd0a-c95a-4c8c-b9cb-85b182475c72-bound-sa-token\") pod \"cert-manager-55b4d954d5-fdpjd\" (UID: \"450dcd0a-c95a-4c8c-b9cb-85b182475c72\") " pod="cert-manager/cert-manager-55b4d954d5-fdpjd" Jul 30 02:34:08.319904 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:08.319875 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/450dcd0a-c95a-4c8c-b9cb-85b182475c72-bound-sa-token\") pod \"cert-manager-55b4d954d5-fdpjd\" (UID: \"450dcd0a-c95a-4c8c-b9cb-85b182475c72\") " pod="cert-manager/cert-manager-55b4d954d5-fdpjd" Jul 30 02:34:08.320026 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:08.319954 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c8c9\" (UniqueName: \"kubernetes.io/projected/450dcd0a-c95a-4c8c-b9cb-85b182475c72-kube-api-access-2c8c9\") pod \"cert-manager-55b4d954d5-fdpjd\" (UID: \"450dcd0a-c95a-4c8c-b9cb-85b182475c72\") " pod="cert-manager/cert-manager-55b4d954d5-fdpjd" Jul 30 02:34:08.402448 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:08.402381 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fdpjd" Jul 30 02:34:08.524563 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:08.524539 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fdpjd"] Jul 30 02:34:08.526735 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:34:08.526706 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod450dcd0a_c95a_4c8c_b9cb_85b182475c72.slice/crio-7e84836abb3673d97da74d4f228090f43927a8e48a8454272bcfe16b88da0b36 WatchSource:0}: Error finding container 7e84836abb3673d97da74d4f228090f43927a8e48a8454272bcfe16b88da0b36: Status 404 returned error can't find the container with id 7e84836abb3673d97da74d4f228090f43927a8e48a8454272bcfe16b88da0b36 Jul 30 02:34:09.503073 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:09.503034 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fdpjd" event={"ID":"450dcd0a-c95a-4c8c-b9cb-85b182475c72","Type":"ContainerStarted","Data":"fbfa8a01a082e0a371e22bbce069f83c042b40df3b68ae5701be1c91d15e3599"} Jul 30 02:34:09.503073 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:09.503076 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fdpjd" event={"ID":"450dcd0a-c95a-4c8c-b9cb-85b182475c72","Type":"ContainerStarted","Data":"7e84836abb3673d97da74d4f228090f43927a8e48a8454272bcfe16b88da0b36"} Jul 30 02:34:09.520360 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:09.520308 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-fdpjd" podStartSLOduration=1.520291243 podStartE2EDuration="1.520291243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:34:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:34:09.518132845 +0000 UTC m=+497.196936839" watchObservedRunningTime="2026-07-30 02:34:09.520291243 +0000 UTC m=+497.199095237" Jul 30 02:34:22.996264 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:22.996218 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:23.096855 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:23.096818 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:23.200214 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:23.200179 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:23.311749 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:23.311643 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:23.792904 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:23.792871 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:24.108441 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.108350 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:24.209274 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.209242 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:24.251990 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.251955 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx"] Jul 30 02:34:24.255025 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.255001 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.257440 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.257379 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 30 02:34:24.257440 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.257391 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 30 02:34:24.257737 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.257609 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 30 02:34:24.257737 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.257721 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 30 02:34:24.258021 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.257899 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fv95l\"" Jul 30 02:34:24.270390 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.270366 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx"] Jul 30 02:34:24.333450 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.333411 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5749fdd-546a-418f-80f3-73bc7330b6d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.333450 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.333456 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5749fdd-546a-418f-80f3-73bc7330b6d7-tmp\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.333736 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.333479 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzwtt\" (UniqueName: \"kubernetes.io/projected/b5749fdd-546a-418f-80f3-73bc7330b6d7-kube-api-access-wzwtt\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.333736 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.333540 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5749fdd-546a-418f-80f3-73bc7330b6d7-manifests\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.333736 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.333601 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5749fdd-546a-418f-80f3-73bc7330b6d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.378715 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.378605 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:24.434357 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.434325 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5749fdd-546a-418f-80f3-73bc7330b6d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.434544 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.434372 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5749fdd-546a-418f-80f3-73bc7330b6d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.434544 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.434394 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5749fdd-546a-418f-80f3-73bc7330b6d7-tmp\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.434544 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.434414 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzwtt\" (UniqueName: \"kubernetes.io/projected/b5749fdd-546a-418f-80f3-73bc7330b6d7-kube-api-access-wzwtt\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.434544 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.434445 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5749fdd-546a-418f-80f3-73bc7330b6d7-manifests\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.434915 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.434895 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5749fdd-546a-418f-80f3-73bc7330b6d7-manifests\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.436887 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.436859 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5749fdd-546a-418f-80f3-73bc7330b6d7-tmp\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.436982 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.436899 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5749fdd-546a-418f-80f3-73bc7330b6d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.436982 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.436967 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5749fdd-546a-418f-80f3-73bc7330b6d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.446055 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.446024 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzwtt\" (UniqueName: \"kubernetes.io/projected/b5749fdd-546a-418f-80f3-73bc7330b6d7-kube-api-access-wzwtt\") pod \"opendatahub-operator-controller-manager-684d7c5d99-8qwsx\" (UID: \"b5749fdd-546a-418f-80f3-73bc7330b6d7\") " pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.479480 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.479442 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:24.565339 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.565311 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:24.580512 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.580446 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:24.688595 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.688565 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:24.710516 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.710496 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx"] Jul 30 02:34:24.713257 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:34:24.713233 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5749fdd_546a_418f_80f3_73bc7330b6d7.slice/crio-ec52e12b353ef24415b23c62f0c8cea505f34db50e4c8b01fbbc2d59578910b0 WatchSource:0}: Error finding container ec52e12b353ef24415b23c62f0c8cea505f34db50e4c8b01fbbc2d59578910b0: Status 404 returned error can't find the container with id ec52e12b353ef24415b23c62f0c8cea505f34db50e4c8b01fbbc2d59578910b0 Jul 30 02:34:24.785399 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.785363 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:24.881465 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.881421 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:24.981063 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:24.980951 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:25.570217 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:25.570177 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" event={"ID":"b5749fdd-546a-418f-80f3-73bc7330b6d7","Type":"ContainerStarted","Data":"ec52e12b353ef24415b23c62f0c8cea505f34db50e4c8b01fbbc2d59578910b0"} Jul 30 02:34:28.585702 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:28.585644 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" event={"ID":"b5749fdd-546a-418f-80f3-73bc7330b6d7","Type":"ContainerStarted","Data":"2db06286fde6e88016d601bf036db76c7d6b7df09c277cf7f2f0381c5f9655bd"} Jul 30 02:34:29.537905 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:29.537870 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:29.591472 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:29.591433 2566 generic.go:356] "Generic (PLEG): container finished" podID="b5749fdd-546a-418f-80f3-73bc7330b6d7" containerID="2db06286fde6e88016d601bf036db76c7d6b7df09c277cf7f2f0381c5f9655bd" exitCode=0 Jul 30 02:34:29.591881 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:29.591513 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" event={"ID":"b5749fdd-546a-418f-80f3-73bc7330b6d7","Type":"ContainerDied","Data":"2db06286fde6e88016d601bf036db76c7d6b7df09c277cf7f2f0381c5f9655bd"} Jul 30 02:34:29.616513 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:29.616479 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:30.597073 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:30.597042 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" event={"ID":"b5749fdd-546a-418f-80f3-73bc7330b6d7","Type":"ContainerStarted","Data":"9411621e12d9f0c714772b9feb265b582824a9ec7da7e509b0ed6a242f11eecb"} Jul 30 02:34:30.597453 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:30.597133 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:30.625851 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:30.625808 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" podStartSLOduration=2.3767608080000002 podStartE2EDuration="6.625792737s" totalImagesPullingTime="3.220878451s" totalInitContainerRuntime="1.028153478s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:34:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:34:30.6239361 +0000 UTC m=+518.302740111" watchObservedRunningTime="2026-07-30 02:34:30.625792737 +0000 UTC m=+518.304596794" Jul 30 02:34:41.603626 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:41.603597 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-684d7c5d99-8qwsx" Jul 30 02:34:43.296036 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:43.295996 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:43.458159 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:43.458116 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:43.988203 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:43.988168 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:44.094607 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:44.094530 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:44.182569 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:44.182531 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:44.451209 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:44.451112 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:44.616845 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:44.616812 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:44.745565 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:44.745475 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:46.518654 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:46.518616 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:46.797730 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:46.797623 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:47.576959 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:47.576926 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:47.642340 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:47.642304 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:48.599737 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:48.599700 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:51.663085 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.663044 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h"] Jul 30 02:34:51.666436 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.666415 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:51.669678 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.669633 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 30 02:34:51.669678 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.669646 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 30 02:34:51.670879 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.670853 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tzr4c\"" Jul 30 02:34:51.670959 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.670883 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 30 02:34:51.671058 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.671041 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 30 02:34:51.671097 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.671089 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 30 02:34:51.682516 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.682494 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h"] Jul 30 02:34:51.726335 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.726313 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4k7s\" (UniqueName: \"kubernetes.io/projected/b24dcdd4-f298-4e4e-924b-3cd2b0405e45-kube-api-access-l4k7s\") pod \"lws-controller-manager-5bb854bdc9-2hb5h\" (UID: \"b24dcdd4-f298-4e4e-924b-3cd2b0405e45\") " pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:51.726441 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.726356 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b24dcdd4-f298-4e4e-924b-3cd2b0405e45-cert\") pod \"lws-controller-manager-5bb854bdc9-2hb5h\" (UID: \"b24dcdd4-f298-4e4e-924b-3cd2b0405e45\") " pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:51.726441 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.726385 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b24dcdd4-f298-4e4e-924b-3cd2b0405e45-metrics-cert\") pod \"lws-controller-manager-5bb854bdc9-2hb5h\" (UID: \"b24dcdd4-f298-4e4e-924b-3cd2b0405e45\") " pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:51.726532 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.726435 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b24dcdd4-f298-4e4e-924b-3cd2b0405e45-manager-config\") pod \"lws-controller-manager-5bb854bdc9-2hb5h\" (UID: \"b24dcdd4-f298-4e4e-924b-3cd2b0405e45\") " pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:51.827566 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.827541 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4k7s\" (UniqueName: \"kubernetes.io/projected/b24dcdd4-f298-4e4e-924b-3cd2b0405e45-kube-api-access-l4k7s\") pod \"lws-controller-manager-5bb854bdc9-2hb5h\" (UID: \"b24dcdd4-f298-4e4e-924b-3cd2b0405e45\") " pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:51.827658 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.827578 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b24dcdd4-f298-4e4e-924b-3cd2b0405e45-cert\") pod \"lws-controller-manager-5bb854bdc9-2hb5h\" (UID: \"b24dcdd4-f298-4e4e-924b-3cd2b0405e45\") " pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:51.827658 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.827596 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b24dcdd4-f298-4e4e-924b-3cd2b0405e45-metrics-cert\") pod \"lws-controller-manager-5bb854bdc9-2hb5h\" (UID: \"b24dcdd4-f298-4e4e-924b-3cd2b0405e45\") " pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:51.827658 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.827618 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b24dcdd4-f298-4e4e-924b-3cd2b0405e45-manager-config\") pod \"lws-controller-manager-5bb854bdc9-2hb5h\" (UID: \"b24dcdd4-f298-4e4e-924b-3cd2b0405e45\") " pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:51.828337 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.828315 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b24dcdd4-f298-4e4e-924b-3cd2b0405e45-manager-config\") pod \"lws-controller-manager-5bb854bdc9-2hb5h\" (UID: \"b24dcdd4-f298-4e4e-924b-3cd2b0405e45\") " pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:51.829970 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.829950 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b24dcdd4-f298-4e4e-924b-3cd2b0405e45-cert\") pod \"lws-controller-manager-5bb854bdc9-2hb5h\" (UID: \"b24dcdd4-f298-4e4e-924b-3cd2b0405e45\") " pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:51.830048 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.829988 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b24dcdd4-f298-4e4e-924b-3cd2b0405e45-metrics-cert\") pod \"lws-controller-manager-5bb854bdc9-2hb5h\" (UID: \"b24dcdd4-f298-4e4e-924b-3cd2b0405e45\") " pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:51.838142 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.838122 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4k7s\" (UniqueName: \"kubernetes.io/projected/b24dcdd4-f298-4e4e-924b-3cd2b0405e45-kube-api-access-l4k7s\") pod \"lws-controller-manager-5bb854bdc9-2hb5h\" (UID: \"b24dcdd4-f298-4e4e-924b-3cd2b0405e45\") " pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:51.976394 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:51.976334 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:52.035080 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.035051 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd"] Jul 30 02:34:52.038631 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.038608 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.041493 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.041471 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 30 02:34:52.041742 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.041566 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 30 02:34:52.041742 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.041574 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 30 02:34:52.041917 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.041904 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qjgdm\"" Jul 30 02:34:52.042054 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.042030 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 30 02:34:52.042202 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.042035 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 30 02:34:52.042315 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.042087 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 30 02:34:52.052252 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.052229 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd"] Jul 30 02:34:52.112212 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.112191 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h"] Jul 30 02:34:52.115172 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:34:52.115144 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb24dcdd4_f298_4e4e_924b_3cd2b0405e45.slice/crio-f4ed9a9c5d74bdf711c7560f9753f89dba8c30899920b8dfe78ee9199c2de725 WatchSource:0}: Error finding container f4ed9a9c5d74bdf711c7560f9753f89dba8c30899920b8dfe78ee9199c2de725: Status 404 returned error can't find the container with id f4ed9a9c5d74bdf711c7560f9753f89dba8c30899920b8dfe78ee9199c2de725 Jul 30 02:34:52.130323 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.130303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c6df948-791a-438a-bbe5-fdcc8108abbd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.130416 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.130342 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c6df948-791a-438a-bbe5-fdcc8108abbd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.130416 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.130361 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2c6df948-791a-438a-bbe5-fdcc8108abbd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.130416 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.130384 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c6df948-791a-438a-bbe5-fdcc8108abbd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.130581 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.130471 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmrcn\" (UniqueName: \"kubernetes.io/projected/2c6df948-791a-438a-bbe5-fdcc8108abbd-kube-api-access-lmrcn\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.130581 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.130516 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2c6df948-791a-438a-bbe5-fdcc8108abbd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.130581 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.130562 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2c6df948-791a-438a-bbe5-fdcc8108abbd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.231433 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.231366 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c6df948-791a-438a-bbe5-fdcc8108abbd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.231433 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.231406 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2c6df948-791a-438a-bbe5-fdcc8108abbd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.231433 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.231429 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c6df948-791a-438a-bbe5-fdcc8108abbd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.231595 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.231462 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmrcn\" (UniqueName: \"kubernetes.io/projected/2c6df948-791a-438a-bbe5-fdcc8108abbd-kube-api-access-lmrcn\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.231595 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.231496 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2c6df948-791a-438a-bbe5-fdcc8108abbd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.231595 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.231528 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2c6df948-791a-438a-bbe5-fdcc8108abbd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.231775 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.231687 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c6df948-791a-438a-bbe5-fdcc8108abbd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.232137 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.232115 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2c6df948-791a-438a-bbe5-fdcc8108abbd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.233688 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.233651 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c6df948-791a-438a-bbe5-fdcc8108abbd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.233834 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.233818 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2c6df948-791a-438a-bbe5-fdcc8108abbd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.234117 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.234091 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2c6df948-791a-438a-bbe5-fdcc8108abbd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.234196 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.234143 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c6df948-791a-438a-bbe5-fdcc8108abbd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.241099 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.241076 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c6df948-791a-438a-bbe5-fdcc8108abbd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.246159 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.246140 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmrcn\" (UniqueName: \"kubernetes.io/projected/2c6df948-791a-438a-bbe5-fdcc8108abbd-kube-api-access-lmrcn\") pod \"istiod-openshift-gateway-5677ccb49b-kctzd\" (UID: \"2c6df948-791a-438a-bbe5-fdcc8108abbd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.270883 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.270856 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:52.351243 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.351224 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:52.383327 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.383304 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:52.490570 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.490544 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd"] Jul 30 02:34:52.493223 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:34:52.493198 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c6df948_791a_438a_bbe5_fdcc8108abbd.slice/crio-5e4b7ef36e02048b4fff3eea2ec14ae75590a136fea190d7ec858226d29b6e76 WatchSource:0}: Error finding container 5e4b7ef36e02048b4fff3eea2ec14ae75590a136fea190d7ec858226d29b6e76: Status 404 returned error can't find the container with id 5e4b7ef36e02048b4fff3eea2ec14ae75590a136fea190d7ec858226d29b6e76 Jul 30 02:34:52.689329 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.689297 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" event={"ID":"b24dcdd4-f298-4e4e-924b-3cd2b0405e45","Type":"ContainerStarted","Data":"f4ed9a9c5d74bdf711c7560f9753f89dba8c30899920b8dfe78ee9199c2de725"} Jul 30 02:34:52.690489 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:52.690468 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" event={"ID":"2c6df948-791a-438a-bbe5-fdcc8108abbd","Type":"ContainerStarted","Data":"5e4b7ef36e02048b4fff3eea2ec14ae75590a136fea190d7ec858226d29b6e76"} Jul 30 02:34:53.331395 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:53.331346 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:53.536084 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:53.536048 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:53.643757 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:53.643650 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:54.719016 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:54.718965 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:54.817548 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:54.817397 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:54.970033 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:54.969958 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:55.057902 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:55.057855 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:55.870263 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:55.870218 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:55.926311 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:55.926276 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:56.052420 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:56.052395 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:56.067213 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:56.067178 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 02:34:56.067297 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:56.067255 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 30 02:34:56.176277 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:56.176217 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:34:56.717289 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:56.717253 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" event={"ID":"b24dcdd4-f298-4e4e-924b-3cd2b0405e45","Type":"ContainerStarted","Data":"aa0ee178de538e834e7b7c7c5fe3dfdad0a65b6f1b811f341272b8e172c3cb55"} Jul 30 02:34:56.717481 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:56.717430 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:34:56.718826 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:56.718799 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" event={"ID":"2c6df948-791a-438a-bbe5-fdcc8108abbd","Type":"ContainerStarted","Data":"dfd066fc1aadc699c4e6d2fde16114e64db5ebd58626837f4760dd9c16b3e1c6"} Jul 30 02:34:56.718931 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:56.718918 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:34:56.734235 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:56.734172 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" podStartSLOduration=1.834045241 podStartE2EDuration="5.734160985s" totalImagesPullingTime="3.900115744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:34:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:34:56.734150647 +0000 UTC m=+544.412954643" watchObservedRunningTime="2026-07-30 02:34:56.734160985 +0000 UTC m=+544.412964992" Jul 30 02:34:56.755529 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:56.755488 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" podStartSLOduration=1.184325319 podStartE2EDuration="4.755475473s" totalImagesPullingTime="3.571150154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:34:56.754435503 +0000 UTC m=+544.433239497" watchObservedRunningTime="2026-07-30 02:34:56.755475473 +0000 UTC m=+544.434279469" Jul 30 02:34:57.725569 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:34:57.725536 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kctzd" Jul 30 02:35:07.725640 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:35:07.725609 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5bb854bdc9-2hb5h" Jul 30 02:35:44.737521 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:35:44.737482 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:35:44.836369 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:35:44.836319 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:35:44.945679 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:35:44.945637 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:35:58.273399 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:35:58.273358 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:35:59.471536 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:35:59.471508 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:00.122670 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:00.122617 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:07.605985 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:07.605947 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:07.663299 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:07.663266 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:07.736038 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:07.736011 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:08.336196 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:08.336159 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:08.436243 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:08.436213 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:18.126856 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:18.126823 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:18.191835 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:18.191804 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:20.065431 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:20.065391 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:23.843714 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:23.843681 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:26.444572 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:26.444535 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:27.048071 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:27.048028 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:27.287043 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:27.287001 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:30.444464 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:30.444417 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:32.087347 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:32.087302 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:34.077953 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:34.077922 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:35.896898 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:35.896860 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:36.869250 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:36.869217 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:36.968467 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:36.968437 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:40.655927 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:40.655882 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:40.784913 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:40.784875 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:42.699352 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:42.699317 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:42.916263 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:42.916224 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:44.891770 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:44.891738 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:44.991131 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:44.991098 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:45.262120 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:45.262090 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:45.361173 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:45.361133 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:49.244119 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:49.244083 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:49.343921 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:49.343890 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:52.385485 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.385456 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44"] Jul 30 02:36:52.388600 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.388584 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:52.392102 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.392081 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 30 02:36:52.392395 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.392347 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kgss5\"" Jul 30 02:36:52.392525 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.392506 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 30 02:36:52.398824 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.398806 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44"] Jul 30 02:36:52.473209 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.473182 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44"] Jul 30 02:36:52.473457 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:36:52.473435 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-br8rb], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" podUID="e0eb3ee4-6b5d-433a-b29c-4708764d1a60" Jul 30 02:36:52.473529 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.473449 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br8rb\" (UniqueName: \"kubernetes.io/projected/e0eb3ee4-6b5d-433a-b29c-4708764d1a60-kube-api-access-br8rb\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2xt44\" (UID: \"e0eb3ee4-6b5d-433a-b29c-4708764d1a60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:52.473529 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.473494 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0eb3ee4-6b5d-433a-b29c-4708764d1a60-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2xt44\" (UID: \"e0eb3ee4-6b5d-433a-b29c-4708764d1a60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:52.574758 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.574730 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br8rb\" (UniqueName: \"kubernetes.io/projected/e0eb3ee4-6b5d-433a-b29c-4708764d1a60-kube-api-access-br8rb\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2xt44\" (UID: \"e0eb3ee4-6b5d-433a-b29c-4708764d1a60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:52.574884 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.574779 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0eb3ee4-6b5d-433a-b29c-4708764d1a60-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2xt44\" (UID: \"e0eb3ee4-6b5d-433a-b29c-4708764d1a60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:52.575117 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.575101 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0eb3ee4-6b5d-433a-b29c-4708764d1a60-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2xt44\" (UID: \"e0eb3ee4-6b5d-433a-b29c-4708764d1a60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:52.583822 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.583791 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br8rb\" (UniqueName: \"kubernetes.io/projected/e0eb3ee4-6b5d-433a-b29c-4708764d1a60-kube-api-access-br8rb\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2xt44\" (UID: \"e0eb3ee4-6b5d-433a-b29c-4708764d1a60\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:52.926863 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.926827 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44"] Jul 30 02:36:52.940017 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:52.939989 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44"] Jul 30 02:36:53.003961 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.003927 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4gw6f"] Jul 30 02:36:53.007734 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.007711 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4gw6f" Jul 30 02:36:53.010989 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.010970 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-pkxld\"" Jul 30 02:36:53.019263 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.019238 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4gw6f"] Jul 30 02:36:53.179568 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.179484 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx2zr\" (UniqueName: \"kubernetes.io/projected/cb82db02-b05d-4a8a-b760-5187383f81c3-kube-api-access-mx2zr\") pod \"limitador-operator-controller-manager-859d7b78bd-4gw6f\" (UID: \"cb82db02-b05d-4a8a-b760-5187383f81c3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4gw6f" Jul 30 02:36:53.206155 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.206127 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:53.208394 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:36:53.208367 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-2xt44\" is forbidden: User \"system:node:ip-10-0-131-5.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-5.ec2.internal' and this object" podUID="e0eb3ee4-6b5d-433a-b29c-4708764d1a60" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:53.212799 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.212782 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:53.215175 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:36:53.215153 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-2xt44\" is forbidden: User \"system:node:ip-10-0-131-5.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-5.ec2.internal' and this object" podUID="e0eb3ee4-6b5d-433a-b29c-4708764d1a60" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:53.280200 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.280173 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx2zr\" (UniqueName: \"kubernetes.io/projected/cb82db02-b05d-4a8a-b760-5187383f81c3-kube-api-access-mx2zr\") pod \"limitador-operator-controller-manager-859d7b78bd-4gw6f\" (UID: \"cb82db02-b05d-4a8a-b760-5187383f81c3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4gw6f" Jul 30 02:36:53.290097 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.290072 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx2zr\" (UniqueName: \"kubernetes.io/projected/cb82db02-b05d-4a8a-b760-5187383f81c3-kube-api-access-mx2zr\") pod \"limitador-operator-controller-manager-859d7b78bd-4gw6f\" (UID: \"cb82db02-b05d-4a8a-b760-5187383f81c3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4gw6f" Jul 30 02:36:53.320821 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.320800 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4gw6f" Jul 30 02:36:53.381597 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.381574 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-br8rb\" (UniqueName: \"kubernetes.io/projected/e0eb3ee4-6b5d-433a-b29c-4708764d1a60-kube-api-access-br8rb\") pod \"e0eb3ee4-6b5d-433a-b29c-4708764d1a60\" (UID: \"e0eb3ee4-6b5d-433a-b29c-4708764d1a60\") " Jul 30 02:36:53.381760 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.381635 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0eb3ee4-6b5d-433a-b29c-4708764d1a60-extensions-socket-volume\") pod \"e0eb3ee4-6b5d-433a-b29c-4708764d1a60\" (UID: \"e0eb3ee4-6b5d-433a-b29c-4708764d1a60\") " Jul 30 02:36:53.382127 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.382075 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0eb3ee4-6b5d-433a-b29c-4708764d1a60-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e0eb3ee4-6b5d-433a-b29c-4708764d1a60" (UID: "e0eb3ee4-6b5d-433a-b29c-4708764d1a60"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 02:36:53.384868 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.384799 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0eb3ee4-6b5d-433a-b29c-4708764d1a60-kube-api-access-br8rb" (OuterVolumeSpecName: "kube-api-access-br8rb") pod "e0eb3ee4-6b5d-433a-b29c-4708764d1a60" (UID: "e0eb3ee4-6b5d-433a-b29c-4708764d1a60"). InnerVolumeSpecName "kube-api-access-br8rb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 02:36:53.451737 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.451685 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4gw6f"] Jul 30 02:36:53.453925 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:36:53.453896 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb82db02_b05d_4a8a_b760_5187383f81c3.slice/crio-fa7668950feae26fa3e5d0f751872fce69df82bc59f60f82b8e68a4aabec6892 WatchSource:0}: Error finding container fa7668950feae26fa3e5d0f751872fce69df82bc59f60f82b8e68a4aabec6892: Status 404 returned error can't find the container with id fa7668950feae26fa3e5d0f751872fce69df82bc59f60f82b8e68a4aabec6892 Jul 30 02:36:53.455824 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.455810 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 02:36:53.482377 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.482352 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-br8rb\" (UniqueName: \"kubernetes.io/projected/e0eb3ee4-6b5d-433a-b29c-4708764d1a60-kube-api-access-br8rb\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:36:53.482377 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:53.482374 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0eb3ee4-6b5d-433a-b29c-4708764d1a60-extensions-socket-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:36:54.211419 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:54.211390 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:54.211602 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:54.211388 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4gw6f" event={"ID":"cb82db02-b05d-4a8a-b760-5187383f81c3","Type":"ContainerStarted","Data":"fa7668950feae26fa3e5d0f751872fce69df82bc59f60f82b8e68a4aabec6892"} Jul 30 02:36:54.213642 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:36:54.213612 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-2xt44\" is forbidden: User \"system:node:ip-10-0-131-5.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-5.ec2.internal' and this object" podUID="e0eb3ee4-6b5d-433a-b29c-4708764d1a60" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:54.219423 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:36:54.219396 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-2xt44\" is forbidden: User \"system:node:ip-10-0-131-5.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-5.ec2.internal' and this object" podUID="e0eb3ee4-6b5d-433a-b29c-4708764d1a60" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2xt44" Jul 30 02:36:54.852823 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:54.852794 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0eb3ee4-6b5d-433a-b29c-4708764d1a60" path="/var/lib/kubelet/pods/e0eb3ee4-6b5d-433a-b29c-4708764d1a60/volumes" Jul 30 02:36:55.838298 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:55.838251 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:55.937363 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:55.937314 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:56.228259 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:56.228224 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4gw6f" event={"ID":"cb82db02-b05d-4a8a-b760-5187383f81c3","Type":"ContainerStarted","Data":"c56a1bf4aa596c282f444151477c53971e51bbd3d69e2d79bfdc77a9f2f77ebc"} Jul 30 02:36:56.228410 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:56.228350 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4gw6f" Jul 30 02:36:56.245188 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:56.245138 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4gw6f" podStartSLOduration=1.6497109380000001 podStartE2EDuration="4.245121574s" totalImagesPullingTime="2.595410636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:36:56.244923664 +0000 UTC m=+663.923727677" watchObservedRunningTime="2026-07-30 02:36:56.245121574 +0000 UTC m=+663.923925569" Jul 30 02:36:58.652932 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:58.652896 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:36:58.757008 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:36:58.756972 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:03.460707 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:03.460626 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:03.561394 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:03.561358 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:06.987337 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:06.987302 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:07.087541 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:07.087502 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:07.235339 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:07.235312 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4gw6f" Jul 30 02:37:13.626658 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:13.626616 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:13.729689 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:13.729630 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:16.821709 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:16.821657 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:16.922380 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:16.922344 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:21.250375 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:21.250325 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:21.305946 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:21.305907 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:21.417623 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:21.417576 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:21.524228 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:21.524126 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:25.597571 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:25.597535 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:25.697858 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:25.697821 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:25.763937 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:25.763906 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:39.708552 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:39.708508 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:39.813810 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:39.813775 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:39.917884 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:39.917846 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:40.560703 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:40.560655 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:40.616632 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:40.616601 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:58.916916 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:58.916838 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:59.016879 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:59.016842 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:37:59.402771 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:37:59.402738 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:11.005872 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:11.005839 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:11.107477 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:11.107429 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:11.206580 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:11.206544 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:11.414864 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:11.414785 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:17.526487 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:17.526445 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:17.635671 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:17.635637 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:20.406484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:20.406448 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:20.506209 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:20.506174 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:53.090047 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:53.090005 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:53.178449 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:53.178410 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:53.589202 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:53.589161 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:53.724076 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:53.724040 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:54.050449 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.050412 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:54.151078 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.151040 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:38:54.158412 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.158388 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4"] Jul 30 02:38:54.161608 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.161594 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4" Jul 30 02:38:54.164426 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.164400 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-f87p6\"" Jul 30 02:38:54.164791 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.164778 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 30 02:38:54.174766 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.174740 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4"] Jul 30 02:38:54.261331 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.261305 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d61232e1-244c-407e-9342-f6ed6bf47758-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-z6xs4\" (UID: \"d61232e1-244c-407e-9342-f6ed6bf47758\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4" Jul 30 02:38:54.261470 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.261354 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7h2d\" (UniqueName: \"kubernetes.io/projected/d61232e1-244c-407e-9342-f6ed6bf47758-kube-api-access-d7h2d\") pod \"payload-pre-processing-687dbb6cf6-z6xs4\" (UID: \"d61232e1-244c-407e-9342-f6ed6bf47758\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4" Jul 30 02:38:54.362467 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.362390 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7h2d\" (UniqueName: \"kubernetes.io/projected/d61232e1-244c-407e-9342-f6ed6bf47758-kube-api-access-d7h2d\") pod \"payload-pre-processing-687dbb6cf6-z6xs4\" (UID: \"d61232e1-244c-407e-9342-f6ed6bf47758\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4" Jul 30 02:38:54.362612 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.362506 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d61232e1-244c-407e-9342-f6ed6bf47758-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-z6xs4\" (UID: \"d61232e1-244c-407e-9342-f6ed6bf47758\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4" Jul 30 02:38:54.363227 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.363207 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d61232e1-244c-407e-9342-f6ed6bf47758-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-z6xs4\" (UID: \"d61232e1-244c-407e-9342-f6ed6bf47758\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4" Jul 30 02:38:54.372927 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.372898 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7h2d\" (UniqueName: \"kubernetes.io/projected/d61232e1-244c-407e-9342-f6ed6bf47758-kube-api-access-d7h2d\") pod \"payload-pre-processing-687dbb6cf6-z6xs4\" (UID: \"d61232e1-244c-407e-9342-f6ed6bf47758\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4" Jul 30 02:38:54.471786 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.471761 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4" Jul 30 02:38:54.596743 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.596639 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4"] Jul 30 02:38:54.599591 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:38:54.599560 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd61232e1_244c_407e_9342_f6ed6bf47758.slice/crio-8cb97710a3f5c46a033e2d73eeeced7312aa5c25a69d8d14741fdc54aacc0372 WatchSource:0}: Error finding container 8cb97710a3f5c46a033e2d73eeeced7312aa5c25a69d8d14741fdc54aacc0372: Status 404 returned error can't find the container with id 8cb97710a3f5c46a033e2d73eeeced7312aa5c25a69d8d14741fdc54aacc0372 Jul 30 02:38:54.753145 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:54.753121 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4" event={"ID":"d61232e1-244c-407e-9342-f6ed6bf47758","Type":"ContainerStarted","Data":"8cb97710a3f5c46a033e2d73eeeced7312aa5c25a69d8d14741fdc54aacc0372"} Jul 30 02:38:56.769743 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:56.769709 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4" event={"ID":"d61232e1-244c-407e-9342-f6ed6bf47758","Type":"ContainerStarted","Data":"fea09b743c8369d68ba626bd86216696863288da42d047f07fb66bd23b60fc9f"} Jul 30 02:38:56.770137 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:56.769822 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4" Jul 30 02:38:56.784533 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:38:56.784494 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4" podStartSLOduration=1.239997094 podStartE2EDuration="2.784482016s" totalImagesPullingTime="1.544484922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:38:56.783567056 +0000 UTC m=+784.462371059" watchObservedRunningTime="2026-07-30 02:38:56.784482016 +0000 UTC m=+784.463286009" Jul 30 02:39:06.070755 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:06.070708 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:39:06.172891 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:06.172862 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:39:07.777061 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:07.777031 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z6xs4" Jul 30 02:39:13.330446 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.330413 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn"] Jul 30 02:39:13.334086 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.334063 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.337455 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.337430 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 30 02:39:13.339278 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.339255 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 30 02:39:13.339404 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.339369 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 30 02:39:13.339478 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.339379 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2mqsd\"" Jul 30 02:39:13.342235 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.342210 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn"] Jul 30 02:39:13.349128 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.349100 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:39:13.412625 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.412601 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grd7h\" (UniqueName: \"kubernetes.io/projected/4e936a43-e407-49d0-a435-07ec83c11999-kube-api-access-grd7h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.412737 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.412639 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e936a43-e407-49d0-a435-07ec83c11999-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.412737 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.412687 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e936a43-e407-49d0-a435-07ec83c11999-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.412737 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.412727 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e936a43-e407-49d0-a435-07ec83c11999-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.412856 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.412747 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e936a43-e407-49d0-a435-07ec83c11999-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.412856 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.412817 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e936a43-e407-49d0-a435-07ec83c11999-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.513950 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.513918 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grd7h\" (UniqueName: \"kubernetes.io/projected/4e936a43-e407-49d0-a435-07ec83c11999-kube-api-access-grd7h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.514073 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.513967 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e936a43-e407-49d0-a435-07ec83c11999-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.514073 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.513994 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e936a43-e407-49d0-a435-07ec83c11999-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.514073 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.514026 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e936a43-e407-49d0-a435-07ec83c11999-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.514225 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.514096 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e936a43-e407-49d0-a435-07ec83c11999-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.514225 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.514197 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e936a43-e407-49d0-a435-07ec83c11999-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.514395 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.514377 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e936a43-e407-49d0-a435-07ec83c11999-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.514479 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.514459 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e936a43-e407-49d0-a435-07ec83c11999-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.514533 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.514517 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e936a43-e407-49d0-a435-07ec83c11999-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.516297 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.516277 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e936a43-e407-49d0-a435-07ec83c11999-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.516506 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.516490 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e936a43-e407-49d0-a435-07ec83c11999-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.521317 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.521295 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grd7h\" (UniqueName: \"kubernetes.io/projected/4e936a43-e407-49d0-a435-07ec83c11999-kube-api-access-grd7h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn\" (UID: \"4e936a43-e407-49d0-a435-07ec83c11999\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.644976 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.644923 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:13.773346 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.773320 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn"] Jul 30 02:39:13.776000 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:39:13.775966 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e936a43_e407_49d0_a435_07ec83c11999.slice/crio-6c440c4e0432b25d55319b08527172cce24d0ed9ecf30706190068f56d8764ad WatchSource:0}: Error finding container 6c440c4e0432b25d55319b08527172cce24d0ed9ecf30706190068f56d8764ad: Status 404 returned error can't find the container with id 6c440c4e0432b25d55319b08527172cce24d0ed9ecf30706190068f56d8764ad Jul 30 02:39:13.851691 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:13.851644 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" event={"ID":"4e936a43-e407-49d0-a435-07ec83c11999","Type":"ContainerStarted","Data":"6c440c4e0432b25d55319b08527172cce24d0ed9ecf30706190068f56d8764ad"} Jul 30 02:39:14.984310 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:14.984276 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:39:15.860614 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:15.860579 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" event={"ID":"4e936a43-e407-49d0-a435-07ec83c11999","Type":"ContainerStarted","Data":"d16976cadc7a3685fa6c919ac21033785bdf7c2d5ed25d78da7b0d05dd91fed8"} Jul 30 02:39:15.876971 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:15.876926 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" podStartSLOduration=1.810938661 podStartE2EDuration="2.876909692s" totalImagesPullingTime="1.065971031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:39:15.875795787 +0000 UTC m=+803.554599785" watchObservedRunningTime="2026-07-30 02:39:15.876909692 +0000 UTC m=+803.555713687" Jul 30 02:39:23.645358 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:23.645323 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:23.645960 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:23.645386 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:23.657834 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:23.657812 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:23.906090 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:23.906009 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn" Jul 30 02:39:46.059078 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:46.059039 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:39:54.054630 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:39:54.054593 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:40:06.421786 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.421756 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv"] Jul 30 02:40:06.425186 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.425166 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.427624 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.427603 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 30 02:40:06.433798 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.433776 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv"] Jul 30 02:40:06.447906 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.447879 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:40:06.547706 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.547657 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbpp9\" (UniqueName: \"kubernetes.io/projected/0bc2a06b-8835-4faf-bd1d-986eae9b769d-kube-api-access-bbpp9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.547869 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.547746 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bc2a06b-8835-4faf-bd1d-986eae9b769d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.547869 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.547793 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bc2a06b-8835-4faf-bd1d-986eae9b769d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.547869 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.547817 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bc2a06b-8835-4faf-bd1d-986eae9b769d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.547869 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.547845 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bc2a06b-8835-4faf-bd1d-986eae9b769d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.548040 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.547879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bc2a06b-8835-4faf-bd1d-986eae9b769d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.649249 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.649207 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bc2a06b-8835-4faf-bd1d-986eae9b769d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.649386 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.649264 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bc2a06b-8835-4faf-bd1d-986eae9b769d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.649386 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.649372 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bc2a06b-8835-4faf-bd1d-986eae9b769d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.649484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.649413 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bc2a06b-8835-4faf-bd1d-986eae9b769d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.649484 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.649439 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bc2a06b-8835-4faf-bd1d-986eae9b769d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.649589 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.649485 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbpp9\" (UniqueName: \"kubernetes.io/projected/0bc2a06b-8835-4faf-bd1d-986eae9b769d-kube-api-access-bbpp9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.649647 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.649582 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bc2a06b-8835-4faf-bd1d-986eae9b769d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.649772 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.649748 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bc2a06b-8835-4faf-bd1d-986eae9b769d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.649968 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.649819 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bc2a06b-8835-4faf-bd1d-986eae9b769d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.651499 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.651468 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bc2a06b-8835-4faf-bd1d-986eae9b769d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.651773 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.651757 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bc2a06b-8835-4faf-bd1d-986eae9b769d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.657087 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.657069 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbpp9\" (UniqueName: \"kubernetes.io/projected/0bc2a06b-8835-4faf-bd1d-986eae9b769d-kube-api-access-bbpp9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv\" (UID: \"0bc2a06b-8835-4faf-bd1d-986eae9b769d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.736006 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.735941 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:06.864523 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:06.864498 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv"] Jul 30 02:40:06.866530 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:40:06.866467 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bc2a06b_8835_4faf_bd1d_986eae9b769d.slice/crio-24a0f6a5bb56e39e4f4f08de0b849780950779c8c56a34574ad6803c6440093c WatchSource:0}: Error finding container 24a0f6a5bb56e39e4f4f08de0b849780950779c8c56a34574ad6803c6440093c: Status 404 returned error can't find the container with id 24a0f6a5bb56e39e4f4f08de0b849780950779c8c56a34574ad6803c6440093c Jul 30 02:40:07.076061 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:07.076030 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" event={"ID":"0bc2a06b-8835-4faf-bd1d-986eae9b769d","Type":"ContainerStarted","Data":"24a0f6a5bb56e39e4f4f08de0b849780950779c8c56a34574ad6803c6440093c"} Jul 30 02:40:08.081472 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:08.081438 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" event={"ID":"0bc2a06b-8835-4faf-bd1d-986eae9b769d","Type":"ContainerStarted","Data":"f7230d5eff30dc86ff116e049ef54ebad8fed1a387f2d88d9dbab68e1f54b79e"} Jul 30 02:40:08.099330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:08.099269 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" podStartSLOduration=1.890079238 podStartE2EDuration="2.099253811s" totalImagesPullingTime="209.174573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:40:08.098939011 +0000 UTC m=+855.777743025" watchObservedRunningTime="2026-07-30 02:40:08.099253811 +0000 UTC m=+855.778057806" Jul 30 02:40:16.625750 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:16.625708 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:40:16.736160 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:16.736133 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:16.736296 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:16.736213 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:16.748583 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:16.748563 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:17.144653 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:17.144624 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv" Jul 30 02:40:23.435841 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.435810 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8"] Jul 30 02:40:23.439178 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.439160 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.441886 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.441866 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 30 02:40:23.448998 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.448977 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8"] Jul 30 02:40:23.460374 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.460347 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:40:23.585922 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.585892 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gsjm\" (UniqueName: \"kubernetes.io/projected/0357639b-a9c5-4f67-a6d4-3fa3208236a7-kube-api-access-2gsjm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.586054 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.585927 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0357639b-a9c5-4f67-a6d4-3fa3208236a7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.586054 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.585969 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0357639b-a9c5-4f67-a6d4-3fa3208236a7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.586054 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.585991 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0357639b-a9c5-4f67-a6d4-3fa3208236a7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.586054 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.586027 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0357639b-a9c5-4f67-a6d4-3fa3208236a7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.586202 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.586104 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0357639b-a9c5-4f67-a6d4-3fa3208236a7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.687501 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.687423 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0357639b-a9c5-4f67-a6d4-3fa3208236a7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.687501 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.687475 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0357639b-a9c5-4f67-a6d4-3fa3208236a7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.687717 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.687532 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0357639b-a9c5-4f67-a6d4-3fa3208236a7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.687717 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.687573 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0357639b-a9c5-4f67-a6d4-3fa3208236a7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.687717 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.687603 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gsjm\" (UniqueName: \"kubernetes.io/projected/0357639b-a9c5-4f67-a6d4-3fa3208236a7-kube-api-access-2gsjm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.687717 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.687628 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0357639b-a9c5-4f67-a6d4-3fa3208236a7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.687987 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.687965 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0357639b-a9c5-4f67-a6d4-3fa3208236a7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.688063 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.688021 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0357639b-a9c5-4f67-a6d4-3fa3208236a7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.688115 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.688079 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0357639b-a9c5-4f67-a6d4-3fa3208236a7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.689843 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.689821 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0357639b-a9c5-4f67-a6d4-3fa3208236a7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.690147 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.690099 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0357639b-a9c5-4f67-a6d4-3fa3208236a7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.696800 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.696780 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gsjm\" (UniqueName: \"kubernetes.io/projected/0357639b-a9c5-4f67-a6d4-3fa3208236a7-kube-api-access-2gsjm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qwbk8\" (UID: \"0357639b-a9c5-4f67-a6d4-3fa3208236a7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.749874 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.749856 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:23.875619 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:23.875594 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8"] Jul 30 02:40:23.878218 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:40:23.878174 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0357639b_a9c5_4f67_a6d4_3fa3208236a7.slice/crio-31729c4e388b38ef17e55e41836203a120ed4d064cbafca802e81b434f6c8296 WatchSource:0}: Error finding container 31729c4e388b38ef17e55e41836203a120ed4d064cbafca802e81b434f6c8296: Status 404 returned error can't find the container with id 31729c4e388b38ef17e55e41836203a120ed4d064cbafca802e81b434f6c8296 Jul 30 02:40:24.166951 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:24.166927 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" event={"ID":"0357639b-a9c5-4f67-a6d4-3fa3208236a7","Type":"ContainerStarted","Data":"c07c1ef75ea36a6580351410af6968da1c019bfa6133fc1c1924228411a339ac"} Jul 30 02:40:24.167053 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:24.166982 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" event={"ID":"0357639b-a9c5-4f67-a6d4-3fa3208236a7","Type":"ContainerStarted","Data":"31729c4e388b38ef17e55e41836203a120ed4d064cbafca802e81b434f6c8296"} Jul 30 02:40:24.186729 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:24.186680 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" podStartSLOduration=0.995500053 podStartE2EDuration="1.186648316s" totalImagesPullingTime="191.148263ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:40:24.184720112 +0000 UTC m=+871.863524105" watchObservedRunningTime="2026-07-30 02:40:24.186648316 +0000 UTC m=+871.865452310" Jul 30 02:40:33.750430 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:33.750393 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:33.750906 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:33.750492 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:33.762651 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:33.762628 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:34.222130 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:34.222046 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qwbk8" Jul 30 02:40:41.272777 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:41.272739 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:40:54.943217 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:54.943176 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:40:55.044162 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:40:55.044118 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:41:02.976568 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:41:02.976522 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:41:26.959693 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:41:26.959647 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:41:40.991757 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:41:40.991724 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:41:41.081958 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:41:41.081928 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:41:42.115550 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:41:42.115513 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:41:42.214167 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:41:42.214137 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:42:09.962241 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:42:09.962208 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:42:10.062381 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:42:10.062351 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:43:02.726086 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:43:02.726048 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:43:02.827038 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:43:02.826996 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:43:47.125284 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:43:47.125192 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:43:48.346834 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:43:48.346798 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:44:22.772498 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:44:22.772463 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:44:22.878585 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:44:22.878535 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:44:22.994813 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:44:22.994778 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:44:26.549749 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:44:26.549711 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:44:28.646939 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:44:28.646896 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:44:28.749978 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:44:28.749944 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:44:28.876656 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:44:28.876619 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:44:29.012379 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:44:29.012340 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:44:34.567077 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:44:34.567038 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:44:34.704757 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:44:34.704717 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:46:11.882727 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:46:11.882695 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:46:13.270508 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:46:13.270468 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:47:12.410083 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:47:12.409940 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:47:12.507888 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:47:12.507836 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:48:40.653198 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:48:40.653166 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:49:34.663001 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:49:34.662965 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:51:24.699110 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:51:24.699071 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:52:46.537185 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:52:46.537109 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:53:11.568408 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:53:11.568374 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:53:47.934004 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:53:47.933960 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:53:49.278649 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:53:49.278610 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:53:53.730490 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:53:53.730455 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:04.663226 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:04.663191 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:28.262212 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:28.262167 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:31.221221 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:31.221178 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:31.324254 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:31.324227 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:31.652320 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:31.652288 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:37.412069 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:37.412029 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:37.512137 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:37.512105 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:37.632794 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:37.632766 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:37.966141 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:37.966107 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:38.065107 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:38.065069 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:40.715865 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:40.715830 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:40.811315 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:40.811287 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:40.979887 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:40.979806 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:41.586362 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:41.586330 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:57:41.689113 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:57:41.689083 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:16.493567 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:16.493531 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:16.554239 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:16.554204 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:19.803600 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:19.803559 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:21.140620 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:21.140574 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:21.320103 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:21.320070 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:21.544123 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:21.544087 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:21.771942 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:21.771907 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:21.866605 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:21.866514 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:21.972102 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:21.972075 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:22.296013 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:22.295983 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:22.388403 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:22.388370 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:24.927523 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:24.927478 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:25.174078 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:25.174048 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:25.353814 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:25.353783 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:25.641030 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:25.640948 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:25.742935 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:25.742905 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:58:25.822584 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:25.822551 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc"] Jul 30 02:58:25.825978 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:25.825958 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" Jul 30 02:58:25.828941 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:25.828917 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-9625a339-dockercfg-qf2fq\"" Jul 30 02:58:25.829053 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:25.828959 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-9625a339\"" Jul 30 02:58:25.840417 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:25.840395 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc"] Jul 30 02:58:25.986993 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:25.986913 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db0d0539-3c50-41fd-b5dc-8cac8de8e150-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc\" (UID: \"db0d0539-3c50-41fd-b5dc-8cac8de8e150\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" Jul 30 02:58:25.986993 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:25.986977 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7fmc\" (UniqueName: \"kubernetes.io/projected/db0d0539-3c50-41fd-b5dc-8cac8de8e150-kube-api-access-n7fmc\") pod \"payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc\" (UID: \"db0d0539-3c50-41fd-b5dc-8cac8de8e150\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" Jul 30 02:58:26.087858 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:26.087826 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db0d0539-3c50-41fd-b5dc-8cac8de8e150-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc\" (UID: \"db0d0539-3c50-41fd-b5dc-8cac8de8e150\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" Jul 30 02:58:26.088011 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:26.087912 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7fmc\" (UniqueName: \"kubernetes.io/projected/db0d0539-3c50-41fd-b5dc-8cac8de8e150-kube-api-access-n7fmc\") pod \"payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc\" (UID: \"db0d0539-3c50-41fd-b5dc-8cac8de8e150\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" Jul 30 02:58:26.088512 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:26.088495 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db0d0539-3c50-41fd-b5dc-8cac8de8e150-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc\" (UID: \"db0d0539-3c50-41fd-b5dc-8cac8de8e150\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" Jul 30 02:58:26.096275 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:26.096251 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7fmc\" (UniqueName: \"kubernetes.io/projected/db0d0539-3c50-41fd-b5dc-8cac8de8e150-kube-api-access-n7fmc\") pod \"payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc\" (UID: \"db0d0539-3c50-41fd-b5dc-8cac8de8e150\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" Jul 30 02:58:26.136210 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:26.136188 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" Jul 30 02:58:26.475334 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:26.475311 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc"] Jul 30 02:58:26.477766 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:58:26.477735 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb0d0539_3c50_41fd_b5dc_8cac8de8e150.slice/crio-e44ea78a815e8f76785fc066cdaba54b3f9839f835e8b371cb910ee81d51b00f WatchSource:0}: Error finding container e44ea78a815e8f76785fc066cdaba54b3f9839f835e8b371cb910ee81d51b00f: Status 404 returned error can't find the container with id e44ea78a815e8f76785fc066cdaba54b3f9839f835e8b371cb910ee81d51b00f Jul 30 02:58:27.017605 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:27.017575 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" event={"ID":"db0d0539-3c50-41fd-b5dc-8cac8de8e150","Type":"ContainerStarted","Data":"1a4181a5634d2a844c556fbb564a02fd127b7fb7ca16ebaed5d007e250176607"} Jul 30 02:58:27.018016 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:27.017623 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" event={"ID":"db0d0539-3c50-41fd-b5dc-8cac8de8e150","Type":"ContainerStarted","Data":"e44ea78a815e8f76785fc066cdaba54b3f9839f835e8b371cb910ee81d51b00f"} Jul 30 02:58:27.018016 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:27.017683 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" Jul 30 02:58:27.034468 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:27.034425 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" podStartSLOduration=2.034412403 podStartE2EDuration="2.034412403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:58:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:58:27.033877171 +0000 UTC m=+1954.712681166" watchObservedRunningTime="2026-07-30 02:58:27.034412403 +0000 UTC m=+1954.713216418" Jul 30 02:58:38.024244 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:58:38.024215 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" Jul 30 02:59:00.314969 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:00.314932 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:00.356040 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:00.356006 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc"] Jul 30 02:59:00.356304 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:00.356258 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" podUID="db0d0539-3c50-41fd-b5dc-8cac8de8e150" containerName="payload-pre-processing" containerID="cri-o://1a4181a5634d2a844c556fbb564a02fd127b7fb7ca16ebaed5d007e250176607" gracePeriod=30 Jul 30 02:59:00.382975 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:00.382940 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:00.439916 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:00.439886 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:00.606250 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:00.606224 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" Jul 30 02:59:00.681898 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:00.681868 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db0d0539-3c50-41fd-b5dc-8cac8de8e150-plugins-config-volume\") pod \"db0d0539-3c50-41fd-b5dc-8cac8de8e150\" (UID: \"db0d0539-3c50-41fd-b5dc-8cac8de8e150\") " Jul 30 02:59:00.682050 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:00.681916 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7fmc\" (UniqueName: \"kubernetes.io/projected/db0d0539-3c50-41fd-b5dc-8cac8de8e150-kube-api-access-n7fmc\") pod \"db0d0539-3c50-41fd-b5dc-8cac8de8e150\" (UID: \"db0d0539-3c50-41fd-b5dc-8cac8de8e150\") " Jul 30 02:59:00.682313 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:00.682288 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db0d0539-3c50-41fd-b5dc-8cac8de8e150-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "db0d0539-3c50-41fd-b5dc-8cac8de8e150" (UID: "db0d0539-3c50-41fd-b5dc-8cac8de8e150"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 02:59:00.684173 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:00.684146 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db0d0539-3c50-41fd-b5dc-8cac8de8e150-kube-api-access-n7fmc" (OuterVolumeSpecName: "kube-api-access-n7fmc") pod "db0d0539-3c50-41fd-b5dc-8cac8de8e150" (UID: "db0d0539-3c50-41fd-b5dc-8cac8de8e150"). InnerVolumeSpecName "kube-api-access-n7fmc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 02:59:00.782970 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:00.782942 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db0d0539-3c50-41fd-b5dc-8cac8de8e150-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:59:00.782970 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:00.782965 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7fmc\" (UniqueName: \"kubernetes.io/projected/db0d0539-3c50-41fd-b5dc-8cac8de8e150-kube-api-access-n7fmc\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:59:01.173185 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:01.173103 2566 generic.go:356] "Generic (PLEG): container finished" podID="db0d0539-3c50-41fd-b5dc-8cac8de8e150" containerID="1a4181a5634d2a844c556fbb564a02fd127b7fb7ca16ebaed5d007e250176607" exitCode=0 Jul 30 02:59:01.173185 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:01.173157 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" Jul 30 02:59:01.173407 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:01.173197 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" event={"ID":"db0d0539-3c50-41fd-b5dc-8cac8de8e150","Type":"ContainerDied","Data":"1a4181a5634d2a844c556fbb564a02fd127b7fb7ca16ebaed5d007e250176607"} Jul 30 02:59:01.173407 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:01.173288 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc" event={"ID":"db0d0539-3c50-41fd-b5dc-8cac8de8e150","Type":"ContainerDied","Data":"e44ea78a815e8f76785fc066cdaba54b3f9839f835e8b371cb910ee81d51b00f"} Jul 30 02:59:01.173407 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:01.173319 2566 scope.go:117] "RemoveContainer" containerID="1a4181a5634d2a844c556fbb564a02fd127b7fb7ca16ebaed5d007e250176607" Jul 30 02:59:01.181791 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:01.181774 2566 scope.go:117] "RemoveContainer" containerID="1a4181a5634d2a844c556fbb564a02fd127b7fb7ca16ebaed5d007e250176607" Jul 30 02:59:01.182053 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:59:01.182033 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a4181a5634d2a844c556fbb564a02fd127b7fb7ca16ebaed5d007e250176607\": container with ID starting with 1a4181a5634d2a844c556fbb564a02fd127b7fb7ca16ebaed5d007e250176607 not found: ID does not exist" containerID="1a4181a5634d2a844c556fbb564a02fd127b7fb7ca16ebaed5d007e250176607" Jul 30 02:59:01.182113 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:01.182061 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a4181a5634d2a844c556fbb564a02fd127b7fb7ca16ebaed5d007e250176607"} err="failed to get container status \"1a4181a5634d2a844c556fbb564a02fd127b7fb7ca16ebaed5d007e250176607\": rpc error: code = NotFound desc = could not find container \"1a4181a5634d2a844c556fbb564a02fd127b7fb7ca16ebaed5d007e250176607\": container with ID starting with 1a4181a5634d2a844c556fbb564a02fd127b7fb7ca16ebaed5d007e250176607 not found: ID does not exist" Jul 30 02:59:01.188909 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:01.188888 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc"] Jul 30 02:59:01.194586 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:01.194566 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9625a339-66758bd764-6wzvc"] Jul 30 02:59:02.852834 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:02.852802 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db0d0539-3c50-41fd-b5dc-8cac8de8e150" path="/var/lib/kubelet/pods/db0d0539-3c50-41fd-b5dc-8cac8de8e150/volumes" Jul 30 02:59:04.971730 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:04.971685 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:05.124427 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:05.124393 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:05.369934 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:05.369896 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:05.590261 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:05.590224 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:05.903473 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:05.903435 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:06.015924 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:06.015894 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:08.198076 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:08.198038 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:08.294999 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:08.294941 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:08.394937 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:08.394885 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:08.496056 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:08.496028 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:08.989051 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:08.988940 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:09.091440 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.091401 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:09.100921 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.100895 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5"] Jul 30 02:59:09.101243 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.101231 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db0d0539-3c50-41fd-b5dc-8cac8de8e150" containerName="payload-pre-processing" Jul 30 02:59:09.101285 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.101244 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="db0d0539-3c50-41fd-b5dc-8cac8de8e150" containerName="payload-pre-processing" Jul 30 02:59:09.101318 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.101302 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="db0d0539-3c50-41fd-b5dc-8cac8de8e150" containerName="payload-pre-processing" Jul 30 02:59:09.105492 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.105476 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" Jul 30 02:59:09.108231 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.108212 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-4fcfd02b\"" Jul 30 02:59:09.108336 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.108212 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-4fcfd02b-dockercfg-6wpm8\"" Jul 30 02:59:09.119364 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.119343 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb"] Jul 30 02:59:09.122678 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.122632 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" Jul 30 02:59:09.123914 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.123879 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5"] Jul 30 02:59:09.132823 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.132802 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb"] Jul 30 02:59:09.258938 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.258911 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f36477d4-4d09-4ac0-bf21-279cf7e4d2d6-plugins-config-volume\") pod \"payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb\" (UID: \"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6\") " pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" Jul 30 02:59:09.259330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.258984 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bd3d565-2653-44a4-9a99-26e1e537c058-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5\" (UID: \"0bd3d565-2653-44a4-9a99-26e1e537c058\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" Jul 30 02:59:09.259330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.259055 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlxbq\" (UniqueName: \"kubernetes.io/projected/0bd3d565-2653-44a4-9a99-26e1e537c058-kube-api-access-zlxbq\") pod \"payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5\" (UID: \"0bd3d565-2653-44a4-9a99-26e1e537c058\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" Jul 30 02:59:09.259330 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.259076 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbkv7\" (UniqueName: \"kubernetes.io/projected/f36477d4-4d09-4ac0-bf21-279cf7e4d2d6-kube-api-access-sbkv7\") pod \"payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb\" (UID: \"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6\") " pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" Jul 30 02:59:09.359820 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.359783 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlxbq\" (UniqueName: \"kubernetes.io/projected/0bd3d565-2653-44a4-9a99-26e1e537c058-kube-api-access-zlxbq\") pod \"payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5\" (UID: \"0bd3d565-2653-44a4-9a99-26e1e537c058\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" Jul 30 02:59:09.359820 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.359818 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbkv7\" (UniqueName: \"kubernetes.io/projected/f36477d4-4d09-4ac0-bf21-279cf7e4d2d6-kube-api-access-sbkv7\") pod \"payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb\" (UID: \"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6\") " pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" Jul 30 02:59:09.360063 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.359861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f36477d4-4d09-4ac0-bf21-279cf7e4d2d6-plugins-config-volume\") pod \"payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb\" (UID: \"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6\") " pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" Jul 30 02:59:09.360063 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.359923 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bd3d565-2653-44a4-9a99-26e1e537c058-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5\" (UID: \"0bd3d565-2653-44a4-9a99-26e1e537c058\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" Jul 30 02:59:09.360714 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.360654 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bd3d565-2653-44a4-9a99-26e1e537c058-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5\" (UID: \"0bd3d565-2653-44a4-9a99-26e1e537c058\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" Jul 30 02:59:09.360714 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.360690 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f36477d4-4d09-4ac0-bf21-279cf7e4d2d6-plugins-config-volume\") pod \"payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb\" (UID: \"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6\") " pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" Jul 30 02:59:09.368712 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.368692 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbkv7\" (UniqueName: \"kubernetes.io/projected/f36477d4-4d09-4ac0-bf21-279cf7e4d2d6-kube-api-access-sbkv7\") pod \"payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb\" (UID: \"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6\") " pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" Jul 30 02:59:09.368835 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.368812 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlxbq\" (UniqueName: \"kubernetes.io/projected/0bd3d565-2653-44a4-9a99-26e1e537c058-kube-api-access-zlxbq\") pod \"payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5\" (UID: \"0bd3d565-2653-44a4-9a99-26e1e537c058\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" Jul 30 02:59:09.415703 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.415676 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" Jul 30 02:59:09.434719 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.434695 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" Jul 30 02:59:09.561324 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.561277 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5"] Jul 30 02:59:09.564762 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:59:09.564735 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bd3d565_2653_44a4_9a99_26e1e537c058.slice/crio-9e2c67da24907a25dc87e9f949063fd5be5aef7f4e359917d77e9560b22e7445 WatchSource:0}: Error finding container 9e2c67da24907a25dc87e9f949063fd5be5aef7f4e359917d77e9560b22e7445: Status 404 returned error can't find the container with id 9e2c67da24907a25dc87e9f949063fd5be5aef7f4e359917d77e9560b22e7445 Jul 30 02:59:09.588252 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:09.588226 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb"] Jul 30 02:59:09.591811 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:59:09.591787 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf36477d4_4d09_4ac0_bf21_279cf7e4d2d6.slice/crio-220613c585f29caee169f59ea36f5391dee2e4c8ca36954c7d5a8136029cb40c WatchSource:0}: Error finding container 220613c585f29caee169f59ea36f5391dee2e4c8ca36954c7d5a8136029cb40c: Status 404 returned error can't find the container with id 220613c585f29caee169f59ea36f5391dee2e4c8ca36954c7d5a8136029cb40c Jul 30 02:59:10.215493 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:10.215456 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" event={"ID":"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6","Type":"ContainerStarted","Data":"94c94cd3ea2a017b2d69418adb655adde8c8669f98ded76c03baaf618d1373e5"} Jul 30 02:59:10.215693 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:10.215514 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" event={"ID":"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6","Type":"ContainerStarted","Data":"220613c585f29caee169f59ea36f5391dee2e4c8ca36954c7d5a8136029cb40c"} Jul 30 02:59:10.215693 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:10.215543 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" Jul 30 02:59:10.216876 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:10.216853 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" event={"ID":"0bd3d565-2653-44a4-9a99-26e1e537c058","Type":"ContainerStarted","Data":"fc2e00c2ac51a9f950e87278cd603eb9aab0e82686d1828ed81ebe2a9736abbf"} Jul 30 02:59:10.216971 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:10.216894 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" event={"ID":"0bd3d565-2653-44a4-9a99-26e1e537c058","Type":"ContainerStarted","Data":"9e2c67da24907a25dc87e9f949063fd5be5aef7f4e359917d77e9560b22e7445"} Jul 30 02:59:10.217030 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:10.216981 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" Jul 30 02:59:10.233985 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:10.233915 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" podStartSLOduration=1.233897016 podStartE2EDuration="1.233897016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:59:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:59:10.231393172 +0000 UTC m=+1997.910197168" watchObservedRunningTime="2026-07-30 02:59:10.233897016 +0000 UTC m=+1997.912701015" Jul 30 02:59:10.248360 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:10.248315 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" podStartSLOduration=1.248303741 podStartE2EDuration="1.248303741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:59:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:59:10.246959417 +0000 UTC m=+1997.925763413" watchObservedRunningTime="2026-07-30 02:59:10.248303741 +0000 UTC m=+1997.927107735" Jul 30 02:59:21.223711 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:21.223680 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" Jul 30 02:59:21.224180 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:21.223840 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" Jul 30 02:59:29.306478 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.306443 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:29.430160 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.430127 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb"] Jul 30 02:59:29.430358 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.430318 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" podUID="f36477d4-4d09-4ac0-bf21-279cf7e4d2d6" containerName="payload-processing" containerID="cri-o://94c94cd3ea2a017b2d69418adb655adde8c8669f98ded76c03baaf618d1373e5" gracePeriod=30 Jul 30 02:59:29.438028 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.437993 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5"] Jul 30 02:59:29.438242 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.438203 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" podUID="0bd3d565-2653-44a4-9a99-26e1e537c058" containerName="payload-pre-processing" containerID="cri-o://fc2e00c2ac51a9f950e87278cd603eb9aab0e82686d1828ed81ebe2a9736abbf" gracePeriod=30 Jul 30 02:59:29.468593 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.468565 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:29.524144 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.524105 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:29.704975 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.704955 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" Jul 30 02:59:29.707801 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.707783 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" Jul 30 02:59:29.739189 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.739164 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlxbq\" (UniqueName: \"kubernetes.io/projected/0bd3d565-2653-44a4-9a99-26e1e537c058-kube-api-access-zlxbq\") pod \"0bd3d565-2653-44a4-9a99-26e1e537c058\" (UID: \"0bd3d565-2653-44a4-9a99-26e1e537c058\") " Jul 30 02:59:29.739298 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.739219 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbkv7\" (UniqueName: \"kubernetes.io/projected/f36477d4-4d09-4ac0-bf21-279cf7e4d2d6-kube-api-access-sbkv7\") pod \"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6\" (UID: \"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6\") " Jul 30 02:59:29.739298 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.739246 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bd3d565-2653-44a4-9a99-26e1e537c058-plugins-config-volume\") pod \"0bd3d565-2653-44a4-9a99-26e1e537c058\" (UID: \"0bd3d565-2653-44a4-9a99-26e1e537c058\") " Jul 30 02:59:29.739408 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.739358 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f36477d4-4d09-4ac0-bf21-279cf7e4d2d6-plugins-config-volume\") pod \"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6\" (UID: \"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6\") " Jul 30 02:59:29.739607 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.739584 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bd3d565-2653-44a4-9a99-26e1e537c058-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0bd3d565-2653-44a4-9a99-26e1e537c058" (UID: "0bd3d565-2653-44a4-9a99-26e1e537c058"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 02:59:29.739747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.739710 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bd3d565-2653-44a4-9a99-26e1e537c058-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:59:29.739747 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.739705 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f36477d4-4d09-4ac0-bf21-279cf7e4d2d6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f36477d4-4d09-4ac0-bf21-279cf7e4d2d6" (UID: "f36477d4-4d09-4ac0-bf21-279cf7e4d2d6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 02:59:29.741355 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.741331 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36477d4-4d09-4ac0-bf21-279cf7e4d2d6-kube-api-access-sbkv7" (OuterVolumeSpecName: "kube-api-access-sbkv7") pod "f36477d4-4d09-4ac0-bf21-279cf7e4d2d6" (UID: "f36477d4-4d09-4ac0-bf21-279cf7e4d2d6"). InnerVolumeSpecName "kube-api-access-sbkv7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 02:59:29.741443 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.741357 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bd3d565-2653-44a4-9a99-26e1e537c058-kube-api-access-zlxbq" (OuterVolumeSpecName: "kube-api-access-zlxbq") pod "0bd3d565-2653-44a4-9a99-26e1e537c058" (UID: "0bd3d565-2653-44a4-9a99-26e1e537c058"). InnerVolumeSpecName "kube-api-access-zlxbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 02:59:29.840091 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.840067 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlxbq\" (UniqueName: \"kubernetes.io/projected/0bd3d565-2653-44a4-9a99-26e1e537c058-kube-api-access-zlxbq\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:59:29.840091 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.840088 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbkv7\" (UniqueName: \"kubernetes.io/projected/f36477d4-4d09-4ac0-bf21-279cf7e4d2d6-kube-api-access-sbkv7\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:59:29.840228 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:29.840098 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f36477d4-4d09-4ac0-bf21-279cf7e4d2d6-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 02:59:30.308978 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.308944 2566 generic.go:356] "Generic (PLEG): container finished" podID="0bd3d565-2653-44a4-9a99-26e1e537c058" containerID="fc2e00c2ac51a9f950e87278cd603eb9aab0e82686d1828ed81ebe2a9736abbf" exitCode=0 Jul 30 02:59:30.309435 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.308985 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" Jul 30 02:59:30.309435 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.309030 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" event={"ID":"0bd3d565-2653-44a4-9a99-26e1e537c058","Type":"ContainerDied","Data":"fc2e00c2ac51a9f950e87278cd603eb9aab0e82686d1828ed81ebe2a9736abbf"} Jul 30 02:59:30.309435 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.309080 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5" event={"ID":"0bd3d565-2653-44a4-9a99-26e1e537c058","Type":"ContainerDied","Data":"9e2c67da24907a25dc87e9f949063fd5be5aef7f4e359917d77e9560b22e7445"} Jul 30 02:59:30.309435 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.309107 2566 scope.go:117] "RemoveContainer" containerID="fc2e00c2ac51a9f950e87278cd603eb9aab0e82686d1828ed81ebe2a9736abbf" Jul 30 02:59:30.310385 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.310360 2566 generic.go:356] "Generic (PLEG): container finished" podID="f36477d4-4d09-4ac0-bf21-279cf7e4d2d6" containerID="94c94cd3ea2a017b2d69418adb655adde8c8669f98ded76c03baaf618d1373e5" exitCode=0 Jul 30 02:59:30.310482 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.310401 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" Jul 30 02:59:30.310482 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.310423 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" event={"ID":"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6","Type":"ContainerDied","Data":"94c94cd3ea2a017b2d69418adb655adde8c8669f98ded76c03baaf618d1373e5"} Jul 30 02:59:30.310482 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.310459 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb" event={"ID":"f36477d4-4d09-4ac0-bf21-279cf7e4d2d6","Type":"ContainerDied","Data":"220613c585f29caee169f59ea36f5391dee2e4c8ca36954c7d5a8136029cb40c"} Jul 30 02:59:30.318951 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.318927 2566 scope.go:117] "RemoveContainer" containerID="fc2e00c2ac51a9f950e87278cd603eb9aab0e82686d1828ed81ebe2a9736abbf" Jul 30 02:59:30.319242 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:59:30.319220 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc2e00c2ac51a9f950e87278cd603eb9aab0e82686d1828ed81ebe2a9736abbf\": container with ID starting with fc2e00c2ac51a9f950e87278cd603eb9aab0e82686d1828ed81ebe2a9736abbf not found: ID does not exist" containerID="fc2e00c2ac51a9f950e87278cd603eb9aab0e82686d1828ed81ebe2a9736abbf" Jul 30 02:59:30.319322 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.319257 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc2e00c2ac51a9f950e87278cd603eb9aab0e82686d1828ed81ebe2a9736abbf"} err="failed to get container status \"fc2e00c2ac51a9f950e87278cd603eb9aab0e82686d1828ed81ebe2a9736abbf\": rpc error: code = NotFound desc = could not find container \"fc2e00c2ac51a9f950e87278cd603eb9aab0e82686d1828ed81ebe2a9736abbf\": container with ID starting with fc2e00c2ac51a9f950e87278cd603eb9aab0e82686d1828ed81ebe2a9736abbf not found: ID does not exist" Jul 30 02:59:30.319322 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.319275 2566 scope.go:117] "RemoveContainer" containerID="94c94cd3ea2a017b2d69418adb655adde8c8669f98ded76c03baaf618d1373e5" Jul 30 02:59:30.329052 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.329032 2566 scope.go:117] "RemoveContainer" containerID="94c94cd3ea2a017b2d69418adb655adde8c8669f98ded76c03baaf618d1373e5" Jul 30 02:59:30.329294 ip-10-0-131-5 kubenswrapper[2566]: E0730 02:59:30.329277 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94c94cd3ea2a017b2d69418adb655adde8c8669f98ded76c03baaf618d1373e5\": container with ID starting with 94c94cd3ea2a017b2d69418adb655adde8c8669f98ded76c03baaf618d1373e5 not found: ID does not exist" containerID="94c94cd3ea2a017b2d69418adb655adde8c8669f98ded76c03baaf618d1373e5" Jul 30 02:59:30.329355 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.329300 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94c94cd3ea2a017b2d69418adb655adde8c8669f98ded76c03baaf618d1373e5"} err="failed to get container status \"94c94cd3ea2a017b2d69418adb655adde8c8669f98ded76c03baaf618d1373e5\": rpc error: code = NotFound desc = could not find container \"94c94cd3ea2a017b2d69418adb655adde8c8669f98ded76c03baaf618d1373e5\": container with ID starting with 94c94cd3ea2a017b2d69418adb655adde8c8669f98ded76c03baaf618d1373e5 not found: ID does not exist" Jul 30 02:59:30.335097 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.335076 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5"] Jul 30 02:59:30.340863 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.340844 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4fcfd02b-d64844b8-wq8d5"] Jul 30 02:59:30.353566 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.353545 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb"] Jul 30 02:59:30.357347 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.357327 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4fcfd02b-6d76cfc98b-v22gb"] Jul 30 02:59:30.851903 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.851873 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bd3d565-2653-44a4-9a99-26e1e537c058" path="/var/lib/kubelet/pods/0bd3d565-2653-44a4-9a99-26e1e537c058/volumes" Jul 30 02:59:30.852230 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:30.852217 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f36477d4-4d09-4ac0-bf21-279cf7e4d2d6" path="/var/lib/kubelet/pods/f36477d4-4d09-4ac0-bf21-279cf7e4d2d6/volumes" Jul 30 02:59:33.572716 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:33.572675 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:34.732649 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:34.732618 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:34.947635 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:34.947601 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:35.159090 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:35.159043 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:35.456400 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:35.456312 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:35.576312 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:35.576274 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:38.338931 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:38.338892 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:38.496975 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:38.496939 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:38.587111 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:38.587083 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:38.690465 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:38.690378 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:39.082048 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.082015 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:39.184277 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.184237 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 02:59:39.190505 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.190480 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn"] Jul 30 02:59:39.190873 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.190860 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bd3d565-2653-44a4-9a99-26e1e537c058" containerName="payload-pre-processing" Jul 30 02:59:39.190913 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.190875 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bd3d565-2653-44a4-9a99-26e1e537c058" containerName="payload-pre-processing" Jul 30 02:59:39.190913 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.190883 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f36477d4-4d09-4ac0-bf21-279cf7e4d2d6" containerName="payload-processing" Jul 30 02:59:39.190913 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.190889 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f36477d4-4d09-4ac0-bf21-279cf7e4d2d6" containerName="payload-processing" Jul 30 02:59:39.191011 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.190958 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f36477d4-4d09-4ac0-bf21-279cf7e4d2d6" containerName="payload-processing" Jul 30 02:59:39.191011 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.190971 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bd3d565-2653-44a4-9a99-26e1e537c058" containerName="payload-pre-processing" Jul 30 02:59:39.195522 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.195506 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" Jul 30 02:59:39.198297 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.198276 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-1ca4664c\"" Jul 30 02:59:39.198481 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.198462 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-1ca4664c-dockercfg-jhdmv\"" Jul 30 02:59:39.205766 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.205744 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn"] Jul 30 02:59:39.322932 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.322903 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0e512e7-b7b4-427d-aee2-d93791c4eef2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn\" (UID: \"f0e512e7-b7b4-427d-aee2-d93791c4eef2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" Jul 30 02:59:39.323076 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.322968 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c58zt\" (UniqueName: \"kubernetes.io/projected/f0e512e7-b7b4-427d-aee2-d93791c4eef2-kube-api-access-c58zt\") pod \"payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn\" (UID: \"f0e512e7-b7b4-427d-aee2-d93791c4eef2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" Jul 30 02:59:39.424427 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.424343 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c58zt\" (UniqueName: \"kubernetes.io/projected/f0e512e7-b7b4-427d-aee2-d93791c4eef2-kube-api-access-c58zt\") pod \"payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn\" (UID: \"f0e512e7-b7b4-427d-aee2-d93791c4eef2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" Jul 30 02:59:39.424427 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.424411 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0e512e7-b7b4-427d-aee2-d93791c4eef2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn\" (UID: \"f0e512e7-b7b4-427d-aee2-d93791c4eef2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" Jul 30 02:59:39.425125 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.425105 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0e512e7-b7b4-427d-aee2-d93791c4eef2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn\" (UID: \"f0e512e7-b7b4-427d-aee2-d93791c4eef2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" Jul 30 02:59:39.431806 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.431777 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c58zt\" (UniqueName: \"kubernetes.io/projected/f0e512e7-b7b4-427d-aee2-d93791c4eef2-kube-api-access-c58zt\") pod \"payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn\" (UID: \"f0e512e7-b7b4-427d-aee2-d93791c4eef2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" Jul 30 02:59:39.506494 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.506467 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" Jul 30 02:59:39.633810 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:39.633736 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn"] Jul 30 02:59:39.636509 ip-10-0-131-5 kubenswrapper[2566]: W0730 02:59:39.636480 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e512e7_b7b4_427d_aee2_d93791c4eef2.slice/crio-daabdadbc229718e873cd1afc1730f0a3d5aecc4255f4fc84b94bc7eef544e3a WatchSource:0}: Error finding container daabdadbc229718e873cd1afc1730f0a3d5aecc4255f4fc84b94bc7eef544e3a: Status 404 returned error can't find the container with id daabdadbc229718e873cd1afc1730f0a3d5aecc4255f4fc84b94bc7eef544e3a Jul 30 02:59:40.369062 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:40.369032 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" event={"ID":"f0e512e7-b7b4-427d-aee2-d93791c4eef2","Type":"ContainerStarted","Data":"d8bbcb00cf44b76dee8ac4d1542ed18c3201980395f1a6913c95afbfdcd85645"} Jul 30 02:59:40.369251 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:40.369084 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" event={"ID":"f0e512e7-b7b4-427d-aee2-d93791c4eef2","Type":"ContainerStarted","Data":"daabdadbc229718e873cd1afc1730f0a3d5aecc4255f4fc84b94bc7eef544e3a"} Jul 30 02:59:40.369251 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:40.369192 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" Jul 30 02:59:40.384717 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:40.384654 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" podStartSLOduration=1.384641141 podStartE2EDuration="1.384641141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 02:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 02:59:40.383304551 +0000 UTC m=+2028.062108546" watchObservedRunningTime="2026-07-30 02:59:40.384641141 +0000 UTC m=+2028.063445136" Jul 30 02:59:51.375070 ip-10-0-131-5 kubenswrapper[2566]: I0730 02:59:51.375042 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" Jul 30 03:00:13.587776 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:13.587738 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:13.631040 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:13.631011 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn"] Jul 30 03:00:13.631271 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:13.631240 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" podUID="f0e512e7-b7b4-427d-aee2-d93791c4eef2" containerName="payload-pre-processing" containerID="cri-o://d8bbcb00cf44b76dee8ac4d1542ed18c3201980395f1a6913c95afbfdcd85645" gracePeriod=30 Jul 30 03:00:13.688122 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:13.688092 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:13.789272 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:13.789066 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:13.874164 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:13.874137 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" Jul 30 03:00:14.018858 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.018829 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c58zt\" (UniqueName: \"kubernetes.io/projected/f0e512e7-b7b4-427d-aee2-d93791c4eef2-kube-api-access-c58zt\") pod \"f0e512e7-b7b4-427d-aee2-d93791c4eef2\" (UID: \"f0e512e7-b7b4-427d-aee2-d93791c4eef2\") " Jul 30 03:00:14.019046 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.018879 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0e512e7-b7b4-427d-aee2-d93791c4eef2-plugins-config-volume\") pod \"f0e512e7-b7b4-427d-aee2-d93791c4eef2\" (UID: \"f0e512e7-b7b4-427d-aee2-d93791c4eef2\") " Jul 30 03:00:14.019281 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.019251 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0e512e7-b7b4-427d-aee2-d93791c4eef2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f0e512e7-b7b4-427d-aee2-d93791c4eef2" (UID: "f0e512e7-b7b4-427d-aee2-d93791c4eef2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:00:14.020894 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.020866 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e512e7-b7b4-427d-aee2-d93791c4eef2-kube-api-access-c58zt" (OuterVolumeSpecName: "kube-api-access-c58zt") pod "f0e512e7-b7b4-427d-aee2-d93791c4eef2" (UID: "f0e512e7-b7b4-427d-aee2-d93791c4eef2"). InnerVolumeSpecName "kube-api-access-c58zt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:00:14.119709 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.119640 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c58zt\" (UniqueName: \"kubernetes.io/projected/f0e512e7-b7b4-427d-aee2-d93791c4eef2-kube-api-access-c58zt\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:00:14.119709 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.119682 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0e512e7-b7b4-427d-aee2-d93791c4eef2-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:00:14.523602 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.523568 2566 generic.go:356] "Generic (PLEG): container finished" podID="f0e512e7-b7b4-427d-aee2-d93791c4eef2" containerID="d8bbcb00cf44b76dee8ac4d1542ed18c3201980395f1a6913c95afbfdcd85645" exitCode=0 Jul 30 03:00:14.523602 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.523604 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" event={"ID":"f0e512e7-b7b4-427d-aee2-d93791c4eef2","Type":"ContainerDied","Data":"d8bbcb00cf44b76dee8ac4d1542ed18c3201980395f1a6913c95afbfdcd85645"} Jul 30 03:00:14.523824 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.523615 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" Jul 30 03:00:14.523824 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.523643 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn" event={"ID":"f0e512e7-b7b4-427d-aee2-d93791c4eef2","Type":"ContainerDied","Data":"daabdadbc229718e873cd1afc1730f0a3d5aecc4255f4fc84b94bc7eef544e3a"} Jul 30 03:00:14.523824 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.523678 2566 scope.go:117] "RemoveContainer" containerID="d8bbcb00cf44b76dee8ac4d1542ed18c3201980395f1a6913c95afbfdcd85645" Jul 30 03:00:14.531903 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.531887 2566 scope.go:117] "RemoveContainer" containerID="d8bbcb00cf44b76dee8ac4d1542ed18c3201980395f1a6913c95afbfdcd85645" Jul 30 03:00:14.532141 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:00:14.532124 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8bbcb00cf44b76dee8ac4d1542ed18c3201980395f1a6913c95afbfdcd85645\": container with ID starting with d8bbcb00cf44b76dee8ac4d1542ed18c3201980395f1a6913c95afbfdcd85645 not found: ID does not exist" containerID="d8bbcb00cf44b76dee8ac4d1542ed18c3201980395f1a6913c95afbfdcd85645" Jul 30 03:00:14.532216 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.532150 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8bbcb00cf44b76dee8ac4d1542ed18c3201980395f1a6913c95afbfdcd85645"} err="failed to get container status \"d8bbcb00cf44b76dee8ac4d1542ed18c3201980395f1a6913c95afbfdcd85645\": rpc error: code = NotFound desc = could not find container \"d8bbcb00cf44b76dee8ac4d1542ed18c3201980395f1a6913c95afbfdcd85645\": container with ID starting with d8bbcb00cf44b76dee8ac4d1542ed18c3201980395f1a6913c95afbfdcd85645 not found: ID does not exist" Jul 30 03:00:14.543880 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.543856 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn"] Jul 30 03:00:14.548368 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.548347 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1ca4664c-6567bf4b8d-rkqcn"] Jul 30 03:00:14.852236 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:14.852160 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0e512e7-b7b4-427d-aee2-d93791c4eef2" path="/var/lib/kubelet/pods/f0e512e7-b7b4-427d-aee2-d93791c4eef2/volumes" Jul 30 03:00:18.265889 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:18.265859 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:18.452866 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:18.452825 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:18.662210 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:18.662122 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:19.137950 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:19.137916 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:19.238867 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:19.238837 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:19.394556 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:19.394463 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:20.770129 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:20.770084 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:26.588079 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:26.588024 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:26.910039 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:26.909956 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:27.010877 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:27.010841 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:27.413078 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:27.413038 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:53.230737 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:53.230635 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:00:59.981929 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:00:59.981897 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:00.046006 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:00.045971 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:00.273471 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:00.273402 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:05.993409 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:05.993376 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:06.045572 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:06.045542 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:16.036030 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:16.035995 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:21.656213 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:21.656176 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:21.745329 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:21.745296 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:21.828024 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:21.827982 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:21.928842 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:21.928765 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:30.296376 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:30.296344 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:32.385476 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:32.385443 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:33.152723 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:33.152690 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:39.397214 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:39.397171 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:39.506013 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:39.505974 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:39.594475 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:39.594447 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:39.708916 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:39.708837 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:40.317707 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:40.317628 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:40.412550 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:40.412517 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:40.511160 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:40.511128 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:41.966918 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:41.966886 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:47.959023 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:47.958916 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:48.150305 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:48.150265 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:48.614034 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:48.614003 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:54.201767 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:54.201721 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:54.301869 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:54.301833 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:54.426766 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:54.426735 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:54.852826 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:54.852774 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:54.933405 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:54.933348 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:57.829699 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:57.829632 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:57.937171 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:57.937140 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:58.031617 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:58.031578 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:58.134192 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:58.134102 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:58.492125 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:58.492047 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:01:58.591784 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:01:58.591751 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:02.356206 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:02.356168 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:02.953915 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:02.953871 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:03.052923 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:03.052890 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:03.391056 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:03.391016 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:09.170913 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:09.170858 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:11.355347 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:11.355313 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:19.939633 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:19.939581 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:20.014213 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:20.014179 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:20.095397 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:20.095360 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:24.597126 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:24.597087 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:24.977478 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:24.977389 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:30.900011 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:30.899968 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:31.027531 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:31.027504 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:31.232185 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:31.232084 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:31.416032 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:31.415996 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:31.556577 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:31.556543 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:31.655934 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:31.655898 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:31.776988 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:31.776947 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:37.370868 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:37.370825 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:37.433501 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:37.433476 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:37.533189 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:37.533160 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:37.633559 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:37.633475 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:37.734683 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:37.734640 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:43.331813 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:43.331774 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:02:43.403237 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:02:43.403209 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:03:24.390380 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:03:24.390344 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:03:41.113753 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:03:41.113715 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:03:41.216166 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:03:41.216136 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:04:35.242770 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:04:35.242732 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:04:35.729516 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:04:35.729482 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:04:35.990656 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:04:35.990579 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:04:38.919811 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:04:38.919766 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:04:39.056487 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:04:39.056453 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:04:39.219972 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:04:39.219936 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:04:39.529844 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:04:39.529808 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:04:40.085848 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:04:40.085815 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:04:40.248724 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:04:40.248688 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:04:43.380806 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:04:43.380772 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:04:48.306161 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:04:48.306078 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:04:57.660124 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:04:57.660075 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:05:38.295387 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:05:38.295354 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:05:38.372419 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:05:38.372380 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:05:38.444568 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:05:38.444533 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:05:42.903458 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:05:42.903418 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:05:46.081077 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:05:46.081031 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:05:51.156065 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:05:51.156020 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:05:58.276649 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:05:58.276606 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:05:58.376481 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:05:58.376426 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:01.109249 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:01.109213 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:27.447598 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:27.447562 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:28.004633 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:28.004597 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:33.572257 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:33.572225 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:33.674927 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:33.674891 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:33.860096 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:33.860019 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:34.077996 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:34.077962 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:35.098043 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:35.098010 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:35.198520 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:35.198466 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:35.300562 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:35.300536 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:35.956747 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:35.956708 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:36.098634 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:36.098596 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:37.352506 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:37.352472 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:43.248431 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:43.248392 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:43.350121 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:43.350091 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:43.893353 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:43.893314 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:50.033056 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:50.033020 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:50.136145 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:50.136110 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:06:50.236966 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:06:50.236927 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:00.745709 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:00.745642 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:00.980890 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:00.980854 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:21.932051 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:21.932019 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:27.973835 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:27.973795 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:28.076306 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:28.076273 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:28.319963 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:28.319932 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:28.624391 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:28.624313 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:28.733116 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:28.733078 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:31.762725 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:31.762683 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:31.849812 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:31.849778 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:31.921502 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:31.921464 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:32.019264 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:32.019181 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:32.441544 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:32.441460 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:32.545707 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:32.545644 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:36.267241 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:36.267211 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:36.586111 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:36.586017 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:36.965141 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:36.965045 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:37.357992 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:37.357951 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:39.912085 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:39.912046 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:39.984324 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:39.984273 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:40.078695 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.078640 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:40.594426 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.594390 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2"] Jul 30 03:07:40.594948 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.594929 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e512e7-b7b4-427d-aee2-d93791c4eef2" containerName="payload-pre-processing" Jul 30 03:07:40.595039 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.594950 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e512e7-b7b4-427d-aee2-d93791c4eef2" containerName="payload-pre-processing" Jul 30 03:07:40.595094 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.595063 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e512e7-b7b4-427d-aee2-d93791c4eef2" containerName="payload-pre-processing" Jul 30 03:07:40.598006 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.597990 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" Jul 30 03:07:40.600280 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.600262 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-cb71ac-dockercfg-44bnm\"" Jul 30 03:07:40.600375 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.600264 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-cb71ac\"" Jul 30 03:07:40.609125 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.609102 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2"] Jul 30 03:07:40.622187 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.622153 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/843ee017-a031-47b6-adcc-ee6c51209e46-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2\" (UID: \"843ee017-a031-47b6-adcc-ee6c51209e46\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" Jul 30 03:07:40.622295 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.622206 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twxfb\" (UniqueName: \"kubernetes.io/projected/843ee017-a031-47b6-adcc-ee6c51209e46-kube-api-access-twxfb\") pod \"payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2\" (UID: \"843ee017-a031-47b6-adcc-ee6c51209e46\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" Jul 30 03:07:40.633565 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.633544 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh"] Jul 30 03:07:40.636849 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.636832 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" Jul 30 03:07:40.638469 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.638446 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:40.646891 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.646871 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh"] Jul 30 03:07:40.701707 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.701655 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:40.722797 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.722772 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twxfb\" (UniqueName: \"kubernetes.io/projected/843ee017-a031-47b6-adcc-ee6c51209e46-kube-api-access-twxfb\") pod \"payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2\" (UID: \"843ee017-a031-47b6-adcc-ee6c51209e46\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" Jul 30 03:07:40.722913 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.722871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7b237ec-a804-4c7d-80bd-8c9d1123f1c8-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh\" (UID: \"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8\") " pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" Jul 30 03:07:40.722981 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.722944 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2nzt\" (UniqueName: \"kubernetes.io/projected/f7b237ec-a804-4c7d-80bd-8c9d1123f1c8-kube-api-access-w2nzt\") pod \"payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh\" (UID: \"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8\") " pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" Jul 30 03:07:40.723036 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.722981 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/843ee017-a031-47b6-adcc-ee6c51209e46-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2\" (UID: \"843ee017-a031-47b6-adcc-ee6c51209e46\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" Jul 30 03:07:40.723598 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.723575 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/843ee017-a031-47b6-adcc-ee6c51209e46-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2\" (UID: \"843ee017-a031-47b6-adcc-ee6c51209e46\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" Jul 30 03:07:40.731641 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.731620 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twxfb\" (UniqueName: \"kubernetes.io/projected/843ee017-a031-47b6-adcc-ee6c51209e46-kube-api-access-twxfb\") pod \"payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2\" (UID: \"843ee017-a031-47b6-adcc-ee6c51209e46\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" Jul 30 03:07:40.824185 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.824162 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7b237ec-a804-4c7d-80bd-8c9d1123f1c8-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh\" (UID: \"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8\") " pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" Jul 30 03:07:40.824291 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.824208 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2nzt\" (UniqueName: \"kubernetes.io/projected/f7b237ec-a804-4c7d-80bd-8c9d1123f1c8-kube-api-access-w2nzt\") pod \"payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh\" (UID: \"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8\") " pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" Jul 30 03:07:40.824870 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.824850 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7b237ec-a804-4c7d-80bd-8c9d1123f1c8-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh\" (UID: \"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8\") " pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" Jul 30 03:07:40.832350 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.832332 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2nzt\" (UniqueName: \"kubernetes.io/projected/f7b237ec-a804-4c7d-80bd-8c9d1123f1c8-kube-api-access-w2nzt\") pod \"payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh\" (UID: \"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8\") " pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" Jul 30 03:07:40.908991 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.908930 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" Jul 30 03:07:40.947794 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:40.947769 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" Jul 30 03:07:41.050745 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:41.050717 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2"] Jul 30 03:07:41.059557 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:07:41.059517 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod843ee017_a031_47b6_adcc_ee6c51209e46.slice/crio-fa0ccb43cce982b27377411c155b6ac64dd85439d1bf5c36aab28ba76afc1dfa WatchSource:0}: Error finding container fa0ccb43cce982b27377411c155b6ac64dd85439d1bf5c36aab28ba76afc1dfa: Status 404 returned error can't find the container with id fa0ccb43cce982b27377411c155b6ac64dd85439d1bf5c36aab28ba76afc1dfa Jul 30 03:07:41.106739 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:07:41.106711 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7b237ec_a804_4c7d_80bd_8c9d1123f1c8.slice/crio-0c63fe9de1ee67b323171651f85f4e05ba45b19ca13a12e4abcbee2cb91dbb01 WatchSource:0}: Error finding container 0c63fe9de1ee67b323171651f85f4e05ba45b19ca13a12e4abcbee2cb91dbb01: Status 404 returned error can't find the container with id 0c63fe9de1ee67b323171651f85f4e05ba45b19ca13a12e4abcbee2cb91dbb01 Jul 30 03:07:41.110951 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:41.110931 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh"] Jul 30 03:07:41.530438 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:41.530396 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" event={"ID":"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8","Type":"ContainerStarted","Data":"1fa40d64d36a595480326192ebb3c02cc0e99a7da88d70828959bc1dc8bfb27d"} Jul 30 03:07:41.530649 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:41.530558 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" Jul 30 03:07:41.530649 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:41.530605 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" event={"ID":"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8","Type":"ContainerStarted","Data":"0c63fe9de1ee67b323171651f85f4e05ba45b19ca13a12e4abcbee2cb91dbb01"} Jul 30 03:07:41.531985 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:41.531963 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" event={"ID":"843ee017-a031-47b6-adcc-ee6c51209e46","Type":"ContainerStarted","Data":"02b50e5e78c66e0b2c34a731b703359e1395b874e79ce8d742f458ab5892fc80"} Jul 30 03:07:41.532115 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:41.531995 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" event={"ID":"843ee017-a031-47b6-adcc-ee6c51209e46","Type":"ContainerStarted","Data":"fa0ccb43cce982b27377411c155b6ac64dd85439d1bf5c36aab28ba76afc1dfa"} Jul 30 03:07:41.532115 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:41.532075 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" Jul 30 03:07:41.554970 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:41.554917 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" podStartSLOduration=1.554900998 podStartE2EDuration="1.554900998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:07:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:07:41.554131005 +0000 UTC m=+2509.232935006" watchObservedRunningTime="2026-07-30 03:07:41.554900998 +0000 UTC m=+2509.233704997" Jul 30 03:07:41.569336 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:41.569295 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" podStartSLOduration=1.569279992 podStartE2EDuration="1.569279992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:07:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:07:41.568184724 +0000 UTC m=+2509.246988720" watchObservedRunningTime="2026-07-30 03:07:41.569279992 +0000 UTC m=+2509.248083988" Jul 30 03:07:44.293574 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:44.293540 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:07:52.538279 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:52.538249 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" Jul 30 03:07:52.538759 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:52.538430 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" Jul 30 03:07:56.437641 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:07:56.437605 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:08:25.978034 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:08:25.977994 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:03.916330 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:03.916294 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:04.901095 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:04.901058 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:05.006592 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:05.006559 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f"] Jul 30 03:09:05.010076 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:05.010046 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f" Jul 30 03:09:05.012703 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:05.012685 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2npk7\"" Jul 30 03:09:05.012808 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:05.012708 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 03:09:05.020864 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:05.020840 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f"] Jul 30 03:09:05.152626 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:05.152543 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c720d5e7-5aa0-4c3a-87d7-25729a5de2a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f\" (UID: \"c720d5e7-5aa0-4c3a-87d7-25729a5de2a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f" Jul 30 03:09:05.253017 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:05.252989 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c720d5e7-5aa0-4c3a-87d7-25729a5de2a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f\" (UID: \"c720d5e7-5aa0-4c3a-87d7-25729a5de2a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f" Jul 30 03:09:05.253605 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:05.253583 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c720d5e7-5aa0-4c3a-87d7-25729a5de2a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f\" (UID: \"c720d5e7-5aa0-4c3a-87d7-25729a5de2a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f" Jul 30 03:09:05.320968 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:05.320942 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f" Jul 30 03:09:05.449292 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:05.449266 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f"] Jul 30 03:09:05.451860 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:09:05.451829 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc720d5e7_5aa0_4c3a_87d7_25729a5de2a8.slice/crio-4dab69cb179b466e73492419fbb629930f3fb1abfae7d00ae9c074918780f25e WatchSource:0}: Error finding container 4dab69cb179b466e73492419fbb629930f3fb1abfae7d00ae9c074918780f25e: Status 404 returned error can't find the container with id 4dab69cb179b466e73492419fbb629930f3fb1abfae7d00ae9c074918780f25e Jul 30 03:09:05.453770 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:05.453750 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 03:09:05.923405 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:05.923368 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f" event={"ID":"c720d5e7-5aa0-4c3a-87d7-25729a5de2a8","Type":"ContainerStarted","Data":"4dab69cb179b466e73492419fbb629930f3fb1abfae7d00ae9c074918780f25e"} Jul 30 03:09:07.935109 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:07.935071 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f" event={"ID":"c720d5e7-5aa0-4c3a-87d7-25729a5de2a8","Type":"ContainerStarted","Data":"64896233ae7748d8ba33809455ffef941e95e07b42e3f1f84becaa2b3c2f9238"} Jul 30 03:09:07.955734 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:07.955655 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f" podStartSLOduration=1.9307903720000001 podStartE2EDuration="3.955642543s" totalImagesPullingTime="2.024852171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:09:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:09:07.955333862 +0000 UTC m=+2595.634137858" watchObservedRunningTime="2026-07-30 03:09:07.955642543 +0000 UTC m=+2595.634446569" Jul 30 03:09:08.940783 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:08.940747 2566 generic.go:356] "Generic (PLEG): container finished" podID="c720d5e7-5aa0-4c3a-87d7-25729a5de2a8" containerID="64896233ae7748d8ba33809455ffef941e95e07b42e3f1f84becaa2b3c2f9238" exitCode=0 Jul 30 03:09:08.941169 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:08.940835 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f" event={"ID":"c720d5e7-5aa0-4c3a-87d7-25729a5de2a8","Type":"ContainerDied","Data":"64896233ae7748d8ba33809455ffef941e95e07b42e3f1f84becaa2b3c2f9238"} Jul 30 03:09:10.074970 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:10.074941 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f" Jul 30 03:09:10.195703 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:10.195677 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c720d5e7-5aa0-4c3a-87d7-25729a5de2a8-maas-api-service-ca\") pod \"c720d5e7-5aa0-4c3a-87d7-25729a5de2a8\" (UID: \"c720d5e7-5aa0-4c3a-87d7-25729a5de2a8\") " Jul 30 03:09:10.196002 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:10.195954 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c720d5e7-5aa0-4c3a-87d7-25729a5de2a8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c720d5e7-5aa0-4c3a-87d7-25729a5de2a8" (UID: "c720d5e7-5aa0-4c3a-87d7-25729a5de2a8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:09:10.296276 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:10.296248 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c720d5e7-5aa0-4c3a-87d7-25729a5de2a8-maas-api-service-ca\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:09:10.950787 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:10.950750 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f" event={"ID":"c720d5e7-5aa0-4c3a-87d7-25729a5de2a8","Type":"ContainerDied","Data":"4dab69cb179b466e73492419fbb629930f3fb1abfae7d00ae9c074918780f25e"} Jul 30 03:09:10.950923 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:10.950791 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f" Jul 30 03:09:10.950923 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:10.950810 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4dab69cb179b466e73492419fbb629930f3fb1abfae7d00ae9c074918780f25e" Jul 30 03:09:15.678158 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:15.678076 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:15.756346 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:15.756306 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:15.855683 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:15.855637 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:20.077454 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:20.077420 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:20.093035 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:20.093009 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f"] Jul 30 03:09:20.095972 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:20.095951 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3b8039-ncc8f"] Jul 30 03:09:20.453869 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:20.453789 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:20.852653 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:20.852620 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c720d5e7-5aa0-4c3a-87d7-25729a5de2a8" path="/var/lib/kubelet/pods/c720d5e7-5aa0-4c3a-87d7-25729a5de2a8/volumes" Jul 30 03:09:26.370840 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:26.370804 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:26.464616 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:26.464580 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:26.627698 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:26.627595 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:26.871725 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:26.871682 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:27.019488 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:27.019459 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl"] Jul 30 03:09:27.020045 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:27.020026 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c720d5e7-5aa0-4c3a-87d7-25729a5de2a8" containerName="revoke-keys" Jul 30 03:09:27.020122 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:27.020047 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c720d5e7-5aa0-4c3a-87d7-25729a5de2a8" containerName="revoke-keys" Jul 30 03:09:27.020161 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:27.020126 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c720d5e7-5aa0-4c3a-87d7-25729a5de2a8" containerName="revoke-keys" Jul 30 03:09:27.023450 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:27.023423 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl" Jul 30 03:09:27.026982 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:27.026966 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 03:09:27.027085 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:27.027039 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2npk7\"" Jul 30 03:09:27.043558 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:27.043531 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl"] Jul 30 03:09:27.133059 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:27.133032 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda331ec-9a24-4bc3-aec3-3320f8352da0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl\" (UID: \"cda331ec-9a24-4bc3-aec3-3320f8352da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl" Jul 30 03:09:27.233520 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:27.233493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda331ec-9a24-4bc3-aec3-3320f8352da0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl\" (UID: \"cda331ec-9a24-4bc3-aec3-3320f8352da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl" Jul 30 03:09:27.234165 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:27.234147 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda331ec-9a24-4bc3-aec3-3320f8352da0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl\" (UID: \"cda331ec-9a24-4bc3-aec3-3320f8352da0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl" Jul 30 03:09:27.333541 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:27.333475 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl" Jul 30 03:09:27.670586 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:27.670562 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl"] Jul 30 03:09:27.672593 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:09:27.672562 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcda331ec_9a24_4bc3_aec3_3320f8352da0.slice/crio-58c911841d4fb8b1c746c7453b60cdf822aebe5466bbe6f5ba87cfb0dfe9807d WatchSource:0}: Error finding container 58c911841d4fb8b1c746c7453b60cdf822aebe5466bbe6f5ba87cfb0dfe9807d: Status 404 returned error can't find the container with id 58c911841d4fb8b1c746c7453b60cdf822aebe5466bbe6f5ba87cfb0dfe9807d Jul 30 03:09:28.029557 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:28.029525 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl" event={"ID":"cda331ec-9a24-4bc3-aec3-3320f8352da0","Type":"ContainerStarted","Data":"cb71f88b32aa481e762e66a50b2d8df78f47cdb595b98f970a9d70bc938fa826"} Jul 30 03:09:28.029693 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:28.029577 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl" event={"ID":"cda331ec-9a24-4bc3-aec3-3320f8352da0","Type":"ContainerStarted","Data":"58c911841d4fb8b1c746c7453b60cdf822aebe5466bbe6f5ba87cfb0dfe9807d"} Jul 30 03:09:28.045139 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:28.045098 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl" podStartSLOduration=2.045086051 podStartE2EDuration="2.045086051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:09:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:09:28.044220719 +0000 UTC m=+2615.723024715" watchObservedRunningTime="2026-07-30 03:09:28.045086051 +0000 UTC m=+2615.723890045" Jul 30 03:09:29.035287 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:29.035255 2566 generic.go:356] "Generic (PLEG): container finished" podID="cda331ec-9a24-4bc3-aec3-3320f8352da0" containerID="cb71f88b32aa481e762e66a50b2d8df78f47cdb595b98f970a9d70bc938fa826" exitCode=0 Jul 30 03:09:29.035681 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:29.035343 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl" event={"ID":"cda331ec-9a24-4bc3-aec3-3320f8352da0","Type":"ContainerDied","Data":"cb71f88b32aa481e762e66a50b2d8df78f47cdb595b98f970a9d70bc938fa826"} Jul 30 03:09:30.168994 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:30.168971 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl" Jul 30 03:09:30.215954 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:30.215913 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:30.257579 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:30.257557 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda331ec-9a24-4bc3-aec3-3320f8352da0-maas-api-service-ca\") pod \"cda331ec-9a24-4bc3-aec3-3320f8352da0\" (UID: \"cda331ec-9a24-4bc3-aec3-3320f8352da0\") " Jul 30 03:09:30.257911 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:30.257892 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cda331ec-9a24-4bc3-aec3-3320f8352da0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cda331ec-9a24-4bc3-aec3-3320f8352da0" (UID: "cda331ec-9a24-4bc3-aec3-3320f8352da0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:09:30.358769 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:30.358707 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda331ec-9a24-4bc3-aec3-3320f8352da0-maas-api-service-ca\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:09:31.045105 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:31.045075 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl" Jul 30 03:09:31.045105 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:31.045090 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl" event={"ID":"cda331ec-9a24-4bc3-aec3-3320f8352da0","Type":"ContainerDied","Data":"58c911841d4fb8b1c746c7453b60cdf822aebe5466bbe6f5ba87cfb0dfe9807d"} Jul 30 03:09:31.045306 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:31.045170 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58c911841d4fb8b1c746c7453b60cdf822aebe5466bbe6f5ba87cfb0dfe9807d" Jul 30 03:09:36.487584 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:36.487549 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:37.421195 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.421154 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:37.446251 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.446218 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh"] Jul 30 03:09:37.446489 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.446461 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" podUID="f7b237ec-a804-4c7d-80bd-8c9d1123f1c8" containerName="payload-processing" containerID="cri-o://1fa40d64d36a595480326192ebb3c02cc0e99a7da88d70828959bc1dc8bfb27d" gracePeriod=30 Jul 30 03:09:37.452949 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.452910 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2"] Jul 30 03:09:37.453318 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.453291 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" podUID="843ee017-a031-47b6-adcc-ee6c51209e46" containerName="payload-pre-processing" containerID="cri-o://02b50e5e78c66e0b2c34a731b703359e1395b874e79ce8d742f458ab5892fc80" gracePeriod=30 Jul 30 03:09:37.490141 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.490114 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:37.554186 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.554157 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:37.727011 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.726985 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" Jul 30 03:09:37.729789 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.729768 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" Jul 30 03:09:37.823599 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.823573 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2nzt\" (UniqueName: \"kubernetes.io/projected/f7b237ec-a804-4c7d-80bd-8c9d1123f1c8-kube-api-access-w2nzt\") pod \"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8\" (UID: \"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8\") " Jul 30 03:09:37.823765 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.823616 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twxfb\" (UniqueName: \"kubernetes.io/projected/843ee017-a031-47b6-adcc-ee6c51209e46-kube-api-access-twxfb\") pod \"843ee017-a031-47b6-adcc-ee6c51209e46\" (UID: \"843ee017-a031-47b6-adcc-ee6c51209e46\") " Jul 30 03:09:37.823765 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.823690 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/843ee017-a031-47b6-adcc-ee6c51209e46-plugins-config-volume\") pod \"843ee017-a031-47b6-adcc-ee6c51209e46\" (UID: \"843ee017-a031-47b6-adcc-ee6c51209e46\") " Jul 30 03:09:37.823765 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.823729 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7b237ec-a804-4c7d-80bd-8c9d1123f1c8-plugins-config-volume\") pod \"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8\" (UID: \"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8\") " Jul 30 03:09:37.824110 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.824079 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/843ee017-a031-47b6-adcc-ee6c51209e46-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "843ee017-a031-47b6-adcc-ee6c51209e46" (UID: "843ee017-a031-47b6-adcc-ee6c51209e46"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:09:37.824110 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.824086 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7b237ec-a804-4c7d-80bd-8c9d1123f1c8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7b237ec-a804-4c7d-80bd-8c9d1123f1c8" (UID: "f7b237ec-a804-4c7d-80bd-8c9d1123f1c8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:09:37.825678 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.825643 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/843ee017-a031-47b6-adcc-ee6c51209e46-kube-api-access-twxfb" (OuterVolumeSpecName: "kube-api-access-twxfb") pod "843ee017-a031-47b6-adcc-ee6c51209e46" (UID: "843ee017-a031-47b6-adcc-ee6c51209e46"). InnerVolumeSpecName "kube-api-access-twxfb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:09:37.825747 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.825688 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7b237ec-a804-4c7d-80bd-8c9d1123f1c8-kube-api-access-w2nzt" (OuterVolumeSpecName: "kube-api-access-w2nzt") pod "f7b237ec-a804-4c7d-80bd-8c9d1123f1c8" (UID: "f7b237ec-a804-4c7d-80bd-8c9d1123f1c8"). InnerVolumeSpecName "kube-api-access-w2nzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:09:37.925104 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.925074 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2nzt\" (UniqueName: \"kubernetes.io/projected/f7b237ec-a804-4c7d-80bd-8c9d1123f1c8-kube-api-access-w2nzt\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:09:37.925104 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.925103 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twxfb\" (UniqueName: \"kubernetes.io/projected/843ee017-a031-47b6-adcc-ee6c51209e46-kube-api-access-twxfb\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:09:37.925301 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.925119 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/843ee017-a031-47b6-adcc-ee6c51209e46-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:09:37.925301 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:37.925133 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7b237ec-a804-4c7d-80bd-8c9d1123f1c8-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:09:38.080370 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.080339 2566 generic.go:356] "Generic (PLEG): container finished" podID="843ee017-a031-47b6-adcc-ee6c51209e46" containerID="02b50e5e78c66e0b2c34a731b703359e1395b874e79ce8d742f458ab5892fc80" exitCode=0 Jul 30 03:09:38.080501 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.080381 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" Jul 30 03:09:38.080501 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.080427 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" event={"ID":"843ee017-a031-47b6-adcc-ee6c51209e46","Type":"ContainerDied","Data":"02b50e5e78c66e0b2c34a731b703359e1395b874e79ce8d742f458ab5892fc80"} Jul 30 03:09:38.080501 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.080483 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2" event={"ID":"843ee017-a031-47b6-adcc-ee6c51209e46","Type":"ContainerDied","Data":"fa0ccb43cce982b27377411c155b6ac64dd85439d1bf5c36aab28ba76afc1dfa"} Jul 30 03:09:38.080693 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.080512 2566 scope.go:117] "RemoveContainer" containerID="02b50e5e78c66e0b2c34a731b703359e1395b874e79ce8d742f458ab5892fc80" Jul 30 03:09:38.081735 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.081711 2566 generic.go:356] "Generic (PLEG): container finished" podID="f7b237ec-a804-4c7d-80bd-8c9d1123f1c8" containerID="1fa40d64d36a595480326192ebb3c02cc0e99a7da88d70828959bc1dc8bfb27d" exitCode=0 Jul 30 03:09:38.081865 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.081751 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" Jul 30 03:09:38.081865 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.081748 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" event={"ID":"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8","Type":"ContainerDied","Data":"1fa40d64d36a595480326192ebb3c02cc0e99a7da88d70828959bc1dc8bfb27d"} Jul 30 03:09:38.081957 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.081866 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh" event={"ID":"f7b237ec-a804-4c7d-80bd-8c9d1123f1c8","Type":"ContainerDied","Data":"0c63fe9de1ee67b323171651f85f4e05ba45b19ca13a12e4abcbee2cb91dbb01"} Jul 30 03:09:38.089960 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.089943 2566 scope.go:117] "RemoveContainer" containerID="02b50e5e78c66e0b2c34a731b703359e1395b874e79ce8d742f458ab5892fc80" Jul 30 03:09:38.090261 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:09:38.090243 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02b50e5e78c66e0b2c34a731b703359e1395b874e79ce8d742f458ab5892fc80\": container with ID starting with 02b50e5e78c66e0b2c34a731b703359e1395b874e79ce8d742f458ab5892fc80 not found: ID does not exist" containerID="02b50e5e78c66e0b2c34a731b703359e1395b874e79ce8d742f458ab5892fc80" Jul 30 03:09:38.090328 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.090268 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02b50e5e78c66e0b2c34a731b703359e1395b874e79ce8d742f458ab5892fc80"} err="failed to get container status \"02b50e5e78c66e0b2c34a731b703359e1395b874e79ce8d742f458ab5892fc80\": rpc error: code = NotFound desc = could not find container \"02b50e5e78c66e0b2c34a731b703359e1395b874e79ce8d742f458ab5892fc80\": container with ID starting with 02b50e5e78c66e0b2c34a731b703359e1395b874e79ce8d742f458ab5892fc80 not found: ID does not exist" Jul 30 03:09:38.090328 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.090284 2566 scope.go:117] "RemoveContainer" containerID="1fa40d64d36a595480326192ebb3c02cc0e99a7da88d70828959bc1dc8bfb27d" Jul 30 03:09:38.097693 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.097643 2566 scope.go:117] "RemoveContainer" containerID="1fa40d64d36a595480326192ebb3c02cc0e99a7da88d70828959bc1dc8bfb27d" Jul 30 03:09:38.097929 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:09:38.097912 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fa40d64d36a595480326192ebb3c02cc0e99a7da88d70828959bc1dc8bfb27d\": container with ID starting with 1fa40d64d36a595480326192ebb3c02cc0e99a7da88d70828959bc1dc8bfb27d not found: ID does not exist" containerID="1fa40d64d36a595480326192ebb3c02cc0e99a7da88d70828959bc1dc8bfb27d" Jul 30 03:09:38.097977 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.097934 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fa40d64d36a595480326192ebb3c02cc0e99a7da88d70828959bc1dc8bfb27d"} err="failed to get container status \"1fa40d64d36a595480326192ebb3c02cc0e99a7da88d70828959bc1dc8bfb27d\": rpc error: code = NotFound desc = could not find container \"1fa40d64d36a595480326192ebb3c02cc0e99a7da88d70828959bc1dc8bfb27d\": container with ID starting with 1fa40d64d36a595480326192ebb3c02cc0e99a7da88d70828959bc1dc8bfb27d not found: ID does not exist" Jul 30 03:09:38.103124 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.103098 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2"] Jul 30 03:09:38.107447 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.107426 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-cb71ac-5d9bf48c64-rx7h2"] Jul 30 03:09:38.117651 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.117631 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh"] Jul 30 03:09:38.121625 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.121601 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-cb71ac-866bbf578b-8dwlh"] Jul 30 03:09:38.853153 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.853122 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="843ee017-a031-47b6-adcc-ee6c51209e46" path="/var/lib/kubelet/pods/843ee017-a031-47b6-adcc-ee6c51209e46/volumes" Jul 30 03:09:38.853466 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:38.853445 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7b237ec-a804-4c7d-80bd-8c9d1123f1c8" path="/var/lib/kubelet/pods/f7b237ec-a804-4c7d-80bd-8c9d1123f1c8/volumes" Jul 30 03:09:42.083989 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:42.083944 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:42.094434 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:42.094410 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl"] Jul 30 03:09:42.098422 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:42.098400 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cb71ac-8rwsl"] Jul 30 03:09:42.482554 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:42.482470 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:42.852489 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:42.852446 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cda331ec-9a24-4bc3-aec3-3320f8352da0" path="/var/lib/kubelet/pods/cda331ec-9a24-4bc3-aec3-3320f8352da0/volumes" Jul 30 03:09:44.741551 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:44.741513 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:48.677761 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:48.677703 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:48.873714 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:48.873653 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:48.993922 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:48.993820 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:49.184475 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:49.184445 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:49.545724 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:49.545688 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:49.904773 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:49.904682 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:50.269599 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:50.269561 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:52.694376 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:52.694311 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:52.797004 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:52.796971 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:52.919807 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:52.919773 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:53.083240 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.083215 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:53.592435 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.592402 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:53.709381 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.709345 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:53.787600 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.787569 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv"] Jul 30 03:09:53.787928 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.787915 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843ee017-a031-47b6-adcc-ee6c51209e46" containerName="payload-pre-processing" Jul 30 03:09:53.787974 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.787929 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="843ee017-a031-47b6-adcc-ee6c51209e46" containerName="payload-pre-processing" Jul 30 03:09:53.787974 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.787940 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cda331ec-9a24-4bc3-aec3-3320f8352da0" containerName="revoke-keys" Jul 30 03:09:53.787974 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.787946 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cda331ec-9a24-4bc3-aec3-3320f8352da0" containerName="revoke-keys" Jul 30 03:09:53.787974 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.787967 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7b237ec-a804-4c7d-80bd-8c9d1123f1c8" containerName="payload-processing" Jul 30 03:09:53.787974 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.787973 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7b237ec-a804-4c7d-80bd-8c9d1123f1c8" containerName="payload-processing" Jul 30 03:09:53.788131 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.788023 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="843ee017-a031-47b6-adcc-ee6c51209e46" containerName="payload-pre-processing" Jul 30 03:09:53.788131 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.788031 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7b237ec-a804-4c7d-80bd-8c9d1123f1c8" containerName="payload-processing" Jul 30 03:09:53.788131 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.788041 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cda331ec-9a24-4bc3-aec3-3320f8352da0" containerName="revoke-keys" Jul 30 03:09:53.793199 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.793181 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" Jul 30 03:09:53.796236 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.796213 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-970a97-dockercfg-88m8q\"" Jul 30 03:09:53.797919 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.797901 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-970a97\"" Jul 30 03:09:53.823420 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.823389 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv"] Jul 30 03:09:53.960360 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.960276 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28m4j\" (UniqueName: \"kubernetes.io/projected/7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7-kube-api-access-28m4j\") pod \"payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv\" (UID: \"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" Jul 30 03:09:53.960360 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:53.960318 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv\" (UID: \"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" Jul 30 03:09:54.060994 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:54.060959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28m4j\" (UniqueName: \"kubernetes.io/projected/7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7-kube-api-access-28m4j\") pod \"payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv\" (UID: \"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" Jul 30 03:09:54.060994 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:54.060998 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv\" (UID: \"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" Jul 30 03:09:54.061621 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:54.061602 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv\" (UID: \"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" Jul 30 03:09:54.070497 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:54.070472 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28m4j\" (UniqueName: \"kubernetes.io/projected/7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7-kube-api-access-28m4j\") pod \"payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv\" (UID: \"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" Jul 30 03:09:54.103421 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:54.103398 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" Jul 30 03:09:54.232866 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:54.232837 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv"] Jul 30 03:09:54.234528 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:09:54.234503 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7efbbd38_6d74_4ca1_9a0f_41c0aaddb6a7.slice/crio-06df198755c568f93434daf27346f2093a4216a49141992b210a930307c12362 WatchSource:0}: Error finding container 06df198755c568f93434daf27346f2093a4216a49141992b210a930307c12362: Status 404 returned error can't find the container with id 06df198755c568f93434daf27346f2093a4216a49141992b210a930307c12362 Jul 30 03:09:55.165737 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:55.165704 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" event={"ID":"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7","Type":"ContainerStarted","Data":"7b0ebad1d6e9e4a968928a1b50c9ae480c0d4a8e7677bdb1c9742d62990be8b5"} Jul 30 03:09:55.166129 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:55.165754 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" event={"ID":"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7","Type":"ContainerStarted","Data":"06df198755c568f93434daf27346f2093a4216a49141992b210a930307c12362"} Jul 30 03:09:55.166129 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:55.165880 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" Jul 30 03:09:55.229395 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:55.229342 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" podStartSLOduration=2.229324271 podStartE2EDuration="2.229324271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:09:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:09:55.225627331 +0000 UTC m=+2642.904431327" watchObservedRunningTime="2026-07-30 03:09:55.229324271 +0000 UTC m=+2642.908128270" Jul 30 03:09:57.262714 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:57.262657 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:57.816370 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:57.816332 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:58.285332 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:58.285291 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:09:58.484655 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:09:58.484624 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:00.933525 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:00.933488 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:01.136943 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:01.136912 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:01.292296 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:01.292228 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:02.302275 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.302241 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:02.523967 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.523933 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:02.615697 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.615609 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf"] Jul 30 03:10:02.619016 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.618996 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" Jul 30 03:10:02.623232 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.623217 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-1a62a0-dockercfg-qx4bl\"" Jul 30 03:10:02.623463 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.623446 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-1a62a0\"" Jul 30 03:10:02.648895 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.648872 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf"] Jul 30 03:10:02.700137 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.700111 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz"] Jul 30 03:10:02.703454 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.703438 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" Jul 30 03:10:02.735974 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.735947 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2h9j\" (UniqueName: \"kubernetes.io/projected/a7457e3a-dc8e-44cf-8370-8c4ab07cedd9-kube-api-access-l2h9j\") pod \"payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf\" (UID: \"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9\") " pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" Jul 30 03:10:02.736088 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.736009 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7457e3a-dc8e-44cf-8370-8c4ab07cedd9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf\" (UID: \"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9\") " pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" Jul 30 03:10:02.755627 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.755595 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz"] Jul 30 03:10:02.837102 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.837060 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2h9j\" (UniqueName: \"kubernetes.io/projected/a7457e3a-dc8e-44cf-8370-8c4ab07cedd9-kube-api-access-l2h9j\") pod \"payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf\" (UID: \"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9\") " pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" Jul 30 03:10:02.837270 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.837219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7457e3a-dc8e-44cf-8370-8c4ab07cedd9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf\" (UID: \"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9\") " pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" Jul 30 03:10:02.837270 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.837261 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08178535-f79e-4922-82e5-b3b8aa44a489-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz\" (UID: \"08178535-f79e-4922-82e5-b3b8aa44a489\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" Jul 30 03:10:02.837349 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.837338 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d27pd\" (UniqueName: \"kubernetes.io/projected/08178535-f79e-4922-82e5-b3b8aa44a489-kube-api-access-d27pd\") pod \"payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz\" (UID: \"08178535-f79e-4922-82e5-b3b8aa44a489\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" Jul 30 03:10:02.837966 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.837945 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7457e3a-dc8e-44cf-8370-8c4ab07cedd9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf\" (UID: \"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9\") " pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" Jul 30 03:10:02.860515 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.860488 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2h9j\" (UniqueName: \"kubernetes.io/projected/a7457e3a-dc8e-44cf-8370-8c4ab07cedd9-kube-api-access-l2h9j\") pod \"payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf\" (UID: \"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9\") " pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" Jul 30 03:10:02.929379 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.929313 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" Jul 30 03:10:02.938395 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.938365 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d27pd\" (UniqueName: \"kubernetes.io/projected/08178535-f79e-4922-82e5-b3b8aa44a489-kube-api-access-d27pd\") pod \"payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz\" (UID: \"08178535-f79e-4922-82e5-b3b8aa44a489\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" Jul 30 03:10:02.938508 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.938440 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08178535-f79e-4922-82e5-b3b8aa44a489-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz\" (UID: \"08178535-f79e-4922-82e5-b3b8aa44a489\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" Jul 30 03:10:02.939069 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.939050 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08178535-f79e-4922-82e5-b3b8aa44a489-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz\" (UID: \"08178535-f79e-4922-82e5-b3b8aa44a489\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" Jul 30 03:10:02.975631 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:02.975604 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d27pd\" (UniqueName: \"kubernetes.io/projected/08178535-f79e-4922-82e5-b3b8aa44a489-kube-api-access-d27pd\") pod \"payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz\" (UID: \"08178535-f79e-4922-82e5-b3b8aa44a489\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" Jul 30 03:10:03.013829 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:03.013353 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" Jul 30 03:10:03.181230 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:03.181203 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz"] Jul 30 03:10:03.183593 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:10:03.183564 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08178535_f79e_4922_82e5_b3b8aa44a489.slice/crio-e882f72512f50a35751977b9689598665232f5a629442f150dfefd54527770ef WatchSource:0}: Error finding container e882f72512f50a35751977b9689598665232f5a629442f150dfefd54527770ef: Status 404 returned error can't find the container with id e882f72512f50a35751977b9689598665232f5a629442f150dfefd54527770ef Jul 30 03:10:03.205014 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:03.204988 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" event={"ID":"08178535-f79e-4922-82e5-b3b8aa44a489","Type":"ContainerStarted","Data":"e882f72512f50a35751977b9689598665232f5a629442f150dfefd54527770ef"} Jul 30 03:10:03.286773 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:03.286751 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf"] Jul 30 03:10:03.288648 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:10:03.288623 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7457e3a_dc8e_44cf_8370_8c4ab07cedd9.slice/crio-2b8e102f10b1d54c89c4e0f7d9bf03975825c61dbb3c1cc712eefc64c5614347 WatchSource:0}: Error finding container 2b8e102f10b1d54c89c4e0f7d9bf03975825c61dbb3c1cc712eefc64c5614347: Status 404 returned error can't find the container with id 2b8e102f10b1d54c89c4e0f7d9bf03975825c61dbb3c1cc712eefc64c5614347 Jul 30 03:10:04.210558 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:04.210525 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" event={"ID":"08178535-f79e-4922-82e5-b3b8aa44a489","Type":"ContainerStarted","Data":"6bac60f18850e95d67fbc9cb3702c2aeb8a8a8ba20088edd20be6dee600564e6"} Jul 30 03:10:04.211035 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:04.210583 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" Jul 30 03:10:04.212085 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:04.212060 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" event={"ID":"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9","Type":"ContainerStarted","Data":"b295325d92a18f05e84eb51a778d8ff05c16c431fdd020aa2d75d0947a133e30"} Jul 30 03:10:04.212218 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:04.212105 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" event={"ID":"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9","Type":"ContainerStarted","Data":"2b8e102f10b1d54c89c4e0f7d9bf03975825c61dbb3c1cc712eefc64c5614347"} Jul 30 03:10:04.212218 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:04.212133 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" Jul 30 03:10:04.256108 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:04.256062 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" podStartSLOduration=2.256047556 podStartE2EDuration="2.256047556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:10:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:10:04.240052611 +0000 UTC m=+2651.918856606" watchObservedRunningTime="2026-07-30 03:10:04.256047556 +0000 UTC m=+2651.934851550" Jul 30 03:10:04.292405 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:04.292364 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" podStartSLOduration=2.2923505029999998 podStartE2EDuration="2.292350503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:10:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:10:04.291764375 +0000 UTC m=+2651.970568371" watchObservedRunningTime="2026-07-30 03:10:04.292350503 +0000 UTC m=+2651.971154498" Jul 30 03:10:05.388768 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:05.388734 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:06.173686 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:06.173636 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" Jul 30 03:10:13.460977 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:13.460943 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6"] Jul 30 03:10:13.464628 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:13.464612 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6" Jul 30 03:10:13.467359 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:13.467329 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 03:10:13.467724 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:13.467706 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2npk7\"" Jul 30 03:10:13.474359 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:13.474334 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6"] Jul 30 03:10:13.522800 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:13.522775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/372328df-49eb-448f-a65e-8606c83a9a4f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-970a97-krpx6\" (UID: \"372328df-49eb-448f-a65e-8606c83a9a4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6" Jul 30 03:10:13.623612 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:13.623586 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/372328df-49eb-448f-a65e-8606c83a9a4f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-970a97-krpx6\" (UID: \"372328df-49eb-448f-a65e-8606c83a9a4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6" Jul 30 03:10:13.624151 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:13.624134 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/372328df-49eb-448f-a65e-8606c83a9a4f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-970a97-krpx6\" (UID: \"372328df-49eb-448f-a65e-8606c83a9a4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6" Jul 30 03:10:13.776743 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:13.776710 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6" Jul 30 03:10:13.909611 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:13.909576 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6"] Jul 30 03:10:13.911042 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:10:13.911017 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372328df_49eb_448f_a65e_8606c83a9a4f.slice/crio-fd6c55be0b28db4973f26351a1f6fc1a00bcb33993960fb96078bff21c7bf12c WatchSource:0}: Error finding container fd6c55be0b28db4973f26351a1f6fc1a00bcb33993960fb96078bff21c7bf12c: Status 404 returned error can't find the container with id fd6c55be0b28db4973f26351a1f6fc1a00bcb33993960fb96078bff21c7bf12c Jul 30 03:10:14.263099 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:14.263062 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6" event={"ID":"372328df-49eb-448f-a65e-8606c83a9a4f","Type":"ContainerStarted","Data":"2717edb03912d4dd6aa89e6ed21cd49a4e735d97f14ed8965cd813c09f47cf26"} Jul 30 03:10:14.263236 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:14.263118 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6" event={"ID":"372328df-49eb-448f-a65e-8606c83a9a4f","Type":"ContainerStarted","Data":"fd6c55be0b28db4973f26351a1f6fc1a00bcb33993960fb96078bff21c7bf12c"} Jul 30 03:10:14.281340 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:14.281264 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6" podStartSLOduration=1.281245005 podStartE2EDuration="1.281245005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:10:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:10:14.28106033 +0000 UTC m=+2661.959864329" watchObservedRunningTime="2026-07-30 03:10:14.281245005 +0000 UTC m=+2661.960049001" Jul 30 03:10:15.218290 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:15.218261 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" Jul 30 03:10:15.218713 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:15.218329 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" Jul 30 03:10:15.269672 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:15.269628 2566 generic.go:356] "Generic (PLEG): container finished" podID="372328df-49eb-448f-a65e-8606c83a9a4f" containerID="2717edb03912d4dd6aa89e6ed21cd49a4e735d97f14ed8965cd813c09f47cf26" exitCode=0 Jul 30 03:10:15.269830 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:15.269714 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6" event={"ID":"372328df-49eb-448f-a65e-8606c83a9a4f","Type":"ContainerDied","Data":"2717edb03912d4dd6aa89e6ed21cd49a4e735d97f14ed8965cd813c09f47cf26"} Jul 30 03:10:16.406963 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:16.406936 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6" Jul 30 03:10:16.444825 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:16.444799 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/372328df-49eb-448f-a65e-8606c83a9a4f-maas-api-service-ca\") pod \"372328df-49eb-448f-a65e-8606c83a9a4f\" (UID: \"372328df-49eb-448f-a65e-8606c83a9a4f\") " Jul 30 03:10:16.445146 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:16.445127 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/372328df-49eb-448f-a65e-8606c83a9a4f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "372328df-49eb-448f-a65e-8606c83a9a4f" (UID: "372328df-49eb-448f-a65e-8606c83a9a4f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:10:16.545807 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:16.545727 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/372328df-49eb-448f-a65e-8606c83a9a4f-maas-api-service-ca\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:10:17.281386 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:17.281362 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6" Jul 30 03:10:17.281568 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:17.281356 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6" event={"ID":"372328df-49eb-448f-a65e-8606c83a9a4f","Type":"ContainerDied","Data":"fd6c55be0b28db4973f26351a1f6fc1a00bcb33993960fb96078bff21c7bf12c"} Jul 30 03:10:17.281568 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:17.281495 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd6c55be0b28db4973f26351a1f6fc1a00bcb33993960fb96078bff21c7bf12c" Jul 30 03:10:29.038800 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.038763 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:29.121010 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.120976 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv"] Jul 30 03:10:29.121260 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.121233 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" podUID="7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7" containerName="payload-pre-processing" containerID="cri-o://7b0ebad1d6e9e4a968928a1b50c9ae480c0d4a8e7677bdb1c9742d62990be8b5" gracePeriod=30 Jul 30 03:10:29.231260 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.231221 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:29.341513 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.341478 2566 generic.go:356] "Generic (PLEG): container finished" podID="7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7" containerID="7b0ebad1d6e9e4a968928a1b50c9ae480c0d4a8e7677bdb1c9742d62990be8b5" exitCode=0 Jul 30 03:10:29.341651 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.341550 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" event={"ID":"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7","Type":"ContainerDied","Data":"7b0ebad1d6e9e4a968928a1b50c9ae480c0d4a8e7677bdb1c9742d62990be8b5"} Jul 30 03:10:29.350933 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.350905 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:29.376643 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.376618 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" Jul 30 03:10:29.457141 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.457115 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28m4j\" (UniqueName: \"kubernetes.io/projected/7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7-kube-api-access-28m4j\") pod \"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7\" (UID: \"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7\") " Jul 30 03:10:29.457299 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.457230 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7-plugins-config-volume\") pod \"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7\" (UID: \"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7\") " Jul 30 03:10:29.457603 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.457576 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7" (UID: "7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:10:29.458987 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.458957 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7-kube-api-access-28m4j" (OuterVolumeSpecName: "kube-api-access-28m4j") pod "7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7" (UID: "7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7"). InnerVolumeSpecName "kube-api-access-28m4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:10:29.558152 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.558083 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:10:29.558152 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:29.558110 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28m4j\" (UniqueName: \"kubernetes.io/projected/7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7-kube-api-access-28m4j\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:10:30.348318 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:30.348282 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" event={"ID":"7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7","Type":"ContainerDied","Data":"06df198755c568f93434daf27346f2093a4216a49141992b210a930307c12362"} Jul 30 03:10:30.348771 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:30.348345 2566 scope.go:117] "RemoveContainer" containerID="7b0ebad1d6e9e4a968928a1b50c9ae480c0d4a8e7677bdb1c9742d62990be8b5" Jul 30 03:10:30.348771 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:30.348296 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv" Jul 30 03:10:30.370961 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:30.370937 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv"] Jul 30 03:10:30.379772 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:30.379750 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-970a97-777f76d57b-pw9jv"] Jul 30 03:10:30.853167 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:30.853134 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7" path="/var/lib/kubelet/pods/7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7/volumes" Jul 30 03:10:33.573216 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:33.573172 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:33.586548 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:33.586522 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6"] Jul 30 03:10:33.588941 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:33.588920 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-970a97-krpx6"] Jul 30 03:10:34.013155 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:34.013127 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:34.852360 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:34.852328 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="372328df-49eb-448f-a65e-8606c83a9a4f" path="/var/lib/kubelet/pods/372328df-49eb-448f-a65e-8606c83a9a4f/volumes" Jul 30 03:10:38.325574 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:38.325539 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:38.481961 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:38.481892 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:40.946164 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:40.946126 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:41.088274 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.088244 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:41.304500 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.304467 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:41.501479 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.501451 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:41.641788 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.641708 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4"] Jul 30 03:10:41.642079 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.642065 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="372328df-49eb-448f-a65e-8606c83a9a4f" containerName="revoke-keys" Jul 30 03:10:41.642135 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.642080 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="372328df-49eb-448f-a65e-8606c83a9a4f" containerName="revoke-keys" Jul 30 03:10:41.642135 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.642102 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7" containerName="payload-pre-processing" Jul 30 03:10:41.642135 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.642109 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7" containerName="payload-pre-processing" Jul 30 03:10:41.642226 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.642169 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7efbbd38-6d74-4ca1-9a0f-41c0aaddb6a7" containerName="payload-pre-processing" Jul 30 03:10:41.642226 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.642181 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="372328df-49eb-448f-a65e-8606c83a9a4f" containerName="revoke-keys" Jul 30 03:10:41.644388 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.644368 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4" Jul 30 03:10:41.647995 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.647974 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2npk7\"" Jul 30 03:10:41.648285 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.648270 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 03:10:41.660430 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.660396 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4"] Jul 30 03:10:41.775196 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.775165 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8371ca-8291-46ff-8d5c-bef5e2679a8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4\" (UID: \"ae8371ca-8291-46ff-8d5c-bef5e2679a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4" Jul 30 03:10:41.876039 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.876011 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8371ca-8291-46ff-8d5c-bef5e2679a8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4\" (UID: \"ae8371ca-8291-46ff-8d5c-bef5e2679a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4" Jul 30 03:10:41.876648 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.876630 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8371ca-8291-46ff-8d5c-bef5e2679a8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4\" (UID: \"ae8371ca-8291-46ff-8d5c-bef5e2679a8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4" Jul 30 03:10:41.955562 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:41.955488 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4" Jul 30 03:10:42.095712 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:42.095683 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4"] Jul 30 03:10:42.099350 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:10:42.099323 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae8371ca_8291_46ff_8d5c_bef5e2679a8d.slice/crio-074382f7fe74415327d0222fa7c4b22bf18dce4b1b683c06f3863750f0730008 WatchSource:0}: Error finding container 074382f7fe74415327d0222fa7c4b22bf18dce4b1b683c06f3863750f0730008: Status 404 returned error can't find the container with id 074382f7fe74415327d0222fa7c4b22bf18dce4b1b683c06f3863750f0730008 Jul 30 03:10:42.407014 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:42.406983 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4" event={"ID":"ae8371ca-8291-46ff-8d5c-bef5e2679a8d","Type":"ContainerStarted","Data":"074382f7fe74415327d0222fa7c4b22bf18dce4b1b683c06f3863750f0730008"} Jul 30 03:10:43.413105 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:43.413073 2566 generic.go:356] "Generic (PLEG): container finished" podID="ae8371ca-8291-46ff-8d5c-bef5e2679a8d" containerID="66032ead3fbb5b51252c8599c9c1a5aa122cd30f931891c42b9d84b9e44fc554" exitCode=0 Jul 30 03:10:43.413485 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:43.413166 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4" event={"ID":"ae8371ca-8291-46ff-8d5c-bef5e2679a8d","Type":"ContainerDied","Data":"66032ead3fbb5b51252c8599c9c1a5aa122cd30f931891c42b9d84b9e44fc554"} Jul 30 03:10:44.559748 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:44.559728 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4" Jul 30 03:10:44.602433 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:44.602409 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8371ca-8291-46ff-8d5c-bef5e2679a8d-maas-api-service-ca\") pod \"ae8371ca-8291-46ff-8d5c-bef5e2679a8d\" (UID: \"ae8371ca-8291-46ff-8d5c-bef5e2679a8d\") " Jul 30 03:10:44.602751 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:44.602729 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae8371ca-8291-46ff-8d5c-bef5e2679a8d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ae8371ca-8291-46ff-8d5c-bef5e2679a8d" (UID: "ae8371ca-8291-46ff-8d5c-bef5e2679a8d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:10:44.703373 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:44.703308 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8371ca-8291-46ff-8d5c-bef5e2679a8d-maas-api-service-ca\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:10:45.423967 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:45.423927 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4" event={"ID":"ae8371ca-8291-46ff-8d5c-bef5e2679a8d","Type":"ContainerDied","Data":"074382f7fe74415327d0222fa7c4b22bf18dce4b1b683c06f3863750f0730008"} Jul 30 03:10:45.423967 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:45.423960 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4" Jul 30 03:10:45.424176 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:45.424012 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="074382f7fe74415327d0222fa7c4b22bf18dce4b1b683c06f3863750f0730008" Jul 30 03:10:57.158144 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.158111 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:57.205738 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.205703 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf"] Jul 30 03:10:57.205980 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.205954 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" podUID="a7457e3a-dc8e-44cf-8370-8c4ab07cedd9" containerName="payload-processing" containerID="cri-o://b295325d92a18f05e84eb51a778d8ff05c16c431fdd020aa2d75d0947a133e30" gracePeriod=30 Jul 30 03:10:57.215404 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.215373 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz"] Jul 30 03:10:57.215628 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.215607 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" podUID="08178535-f79e-4922-82e5-b3b8aa44a489" containerName="payload-pre-processing" containerID="cri-o://6bac60f18850e95d67fbc9cb3702c2aeb8a8a8ba20088edd20be6dee600564e6" gracePeriod=30 Jul 30 03:10:57.248910 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.248880 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:57.349403 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.349330 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:10:57.466223 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.466194 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" Jul 30 03:10:57.483863 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.483766 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" Jul 30 03:10:57.484143 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.484114 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" event={"ID":"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9","Type":"ContainerDied","Data":"b295325d92a18f05e84eb51a778d8ff05c16c431fdd020aa2d75d0947a133e30"} Jul 30 03:10:57.484253 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.484177 2566 scope.go:117] "RemoveContainer" containerID="b295325d92a18f05e84eb51a778d8ff05c16c431fdd020aa2d75d0947a133e30" Jul 30 03:10:57.484312 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.484279 2566 generic.go:356] "Generic (PLEG): container finished" podID="a7457e3a-dc8e-44cf-8370-8c4ab07cedd9" containerID="b295325d92a18f05e84eb51a778d8ff05c16c431fdd020aa2d75d0947a133e30" exitCode=0 Jul 30 03:10:57.484623 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.484422 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf" event={"ID":"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9","Type":"ContainerDied","Data":"2b8e102f10b1d54c89c4e0f7d9bf03975825c61dbb3c1cc712eefc64c5614347"} Jul 30 03:10:57.486854 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.486830 2566 generic.go:356] "Generic (PLEG): container finished" podID="08178535-f79e-4922-82e5-b3b8aa44a489" containerID="6bac60f18850e95d67fbc9cb3702c2aeb8a8a8ba20088edd20be6dee600564e6" exitCode=0 Jul 30 03:10:57.486977 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.486886 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" event={"ID":"08178535-f79e-4922-82e5-b3b8aa44a489","Type":"ContainerDied","Data":"6bac60f18850e95d67fbc9cb3702c2aeb8a8a8ba20088edd20be6dee600564e6"} Jul 30 03:10:57.495677 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.495641 2566 scope.go:117] "RemoveContainer" containerID="b295325d92a18f05e84eb51a778d8ff05c16c431fdd020aa2d75d0947a133e30" Jul 30 03:10:57.495931 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:10:57.495904 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b295325d92a18f05e84eb51a778d8ff05c16c431fdd020aa2d75d0947a133e30\": container with ID starting with b295325d92a18f05e84eb51a778d8ff05c16c431fdd020aa2d75d0947a133e30 not found: ID does not exist" containerID="b295325d92a18f05e84eb51a778d8ff05c16c431fdd020aa2d75d0947a133e30" Jul 30 03:10:57.495979 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.495943 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b295325d92a18f05e84eb51a778d8ff05c16c431fdd020aa2d75d0947a133e30"} err="failed to get container status \"b295325d92a18f05e84eb51a778d8ff05c16c431fdd020aa2d75d0947a133e30\": rpc error: code = NotFound desc = could not find container \"b295325d92a18f05e84eb51a778d8ff05c16c431fdd020aa2d75d0947a133e30\": container with ID starting with b295325d92a18f05e84eb51a778d8ff05c16c431fdd020aa2d75d0947a133e30 not found: ID does not exist" Jul 30 03:10:57.496927 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.496913 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" Jul 30 03:10:57.523883 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.523848 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7457e3a-dc8e-44cf-8370-8c4ab07cedd9-plugins-config-volume\") pod \"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9\" (UID: \"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9\") " Jul 30 03:10:57.524035 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.523929 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2h9j\" (UniqueName: \"kubernetes.io/projected/a7457e3a-dc8e-44cf-8370-8c4ab07cedd9-kube-api-access-l2h9j\") pod \"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9\" (UID: \"a7457e3a-dc8e-44cf-8370-8c4ab07cedd9\") " Jul 30 03:10:57.524210 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.524187 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7457e3a-dc8e-44cf-8370-8c4ab07cedd9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7457e3a-dc8e-44cf-8370-8c4ab07cedd9" (UID: "a7457e3a-dc8e-44cf-8370-8c4ab07cedd9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:10:57.525999 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.525978 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7457e3a-dc8e-44cf-8370-8c4ab07cedd9-kube-api-access-l2h9j" (OuterVolumeSpecName: "kube-api-access-l2h9j") pod "a7457e3a-dc8e-44cf-8370-8c4ab07cedd9" (UID: "a7457e3a-dc8e-44cf-8370-8c4ab07cedd9"). InnerVolumeSpecName "kube-api-access-l2h9j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:10:57.625383 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.625357 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d27pd\" (UniqueName: \"kubernetes.io/projected/08178535-f79e-4922-82e5-b3b8aa44a489-kube-api-access-d27pd\") pod \"08178535-f79e-4922-82e5-b3b8aa44a489\" (UID: \"08178535-f79e-4922-82e5-b3b8aa44a489\") " Jul 30 03:10:57.625536 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.625425 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08178535-f79e-4922-82e5-b3b8aa44a489-plugins-config-volume\") pod \"08178535-f79e-4922-82e5-b3b8aa44a489\" (UID: \"08178535-f79e-4922-82e5-b3b8aa44a489\") " Jul 30 03:10:57.625585 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.625567 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7457e3a-dc8e-44cf-8370-8c4ab07cedd9-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:10:57.625585 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.625579 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2h9j\" (UniqueName: \"kubernetes.io/projected/a7457e3a-dc8e-44cf-8370-8c4ab07cedd9-kube-api-access-l2h9j\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:10:57.625853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.625829 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08178535-f79e-4922-82e5-b3b8aa44a489-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "08178535-f79e-4922-82e5-b3b8aa44a489" (UID: "08178535-f79e-4922-82e5-b3b8aa44a489"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:10:57.627500 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.627472 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08178535-f79e-4922-82e5-b3b8aa44a489-kube-api-access-d27pd" (OuterVolumeSpecName: "kube-api-access-d27pd") pod "08178535-f79e-4922-82e5-b3b8aa44a489" (UID: "08178535-f79e-4922-82e5-b3b8aa44a489"). InnerVolumeSpecName "kube-api-access-d27pd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:10:57.726308 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.726237 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08178535-f79e-4922-82e5-b3b8aa44a489-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:10:57.726308 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.726260 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d27pd\" (UniqueName: \"kubernetes.io/projected/08178535-f79e-4922-82e5-b3b8aa44a489-kube-api-access-d27pd\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:10:57.804445 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.804419 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf"] Jul 30 03:10:57.809426 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:57.809395 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1a62a0-5458c54848-47qxf"] Jul 30 03:10:58.492822 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:58.492749 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" Jul 30 03:10:58.492822 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:58.492782 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz" event={"ID":"08178535-f79e-4922-82e5-b3b8aa44a489","Type":"ContainerDied","Data":"e882f72512f50a35751977b9689598665232f5a629442f150dfefd54527770ef"} Jul 30 03:10:58.493294 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:58.492838 2566 scope.go:117] "RemoveContainer" containerID="6bac60f18850e95d67fbc9cb3702c2aeb8a8a8ba20088edd20be6dee600564e6" Jul 30 03:10:58.514736 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:58.514711 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz"] Jul 30 03:10:58.518234 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:58.518212 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1a62a0-6954bddbd8-mm7hz"] Jul 30 03:10:58.858577 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:58.858544 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08178535-f79e-4922-82e5-b3b8aa44a489" path="/var/lib/kubelet/pods/08178535-f79e-4922-82e5-b3b8aa44a489/volumes" Jul 30 03:10:58.858907 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:10:58.858893 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7457e3a-dc8e-44cf-8370-8c4ab07cedd9" path="/var/lib/kubelet/pods/a7457e3a-dc8e-44cf-8370-8c4ab07cedd9/volumes" Jul 30 03:11:01.714462 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:01.714423 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4"] Jul 30 03:11:01.719378 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:01.719347 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:01.719728 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:01.719708 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1a62a0-gknc4"] Jul 30 03:11:02.122127 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:02.122082 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:02.852825 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:02.852788 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae8371ca-8291-46ff-8d5c-bef5e2679a8d" path="/var/lib/kubelet/pods/ae8371ca-8291-46ff-8d5c-bef5e2679a8d/volumes" Jul 30 03:11:08.083392 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:08.083353 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:08.193465 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:08.193438 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:08.391604 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:08.391524 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:08.593117 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:08.593079 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:08.958860 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:08.958827 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:09.278285 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:09.278239 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:10.034382 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:10.034338 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:12.368368 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:12.368328 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:12.459963 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:12.459932 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:12.580361 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:12.580321 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:12.926967 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:12.926936 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:13.200169 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.200084 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:13.292816 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.292780 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:13.361195 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.361165 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx"] Jul 30 03:11:13.361504 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.361492 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7457e3a-dc8e-44cf-8370-8c4ab07cedd9" containerName="payload-processing" Jul 30 03:11:13.361549 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.361505 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7457e3a-dc8e-44cf-8370-8c4ab07cedd9" containerName="payload-processing" Jul 30 03:11:13.361549 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.361522 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae8371ca-8291-46ff-8d5c-bef5e2679a8d" containerName="revoke-keys" Jul 30 03:11:13.361549 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.361528 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae8371ca-8291-46ff-8d5c-bef5e2679a8d" containerName="revoke-keys" Jul 30 03:11:13.361549 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.361546 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08178535-f79e-4922-82e5-b3b8aa44a489" containerName="payload-pre-processing" Jul 30 03:11:13.361713 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.361552 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="08178535-f79e-4922-82e5-b3b8aa44a489" containerName="payload-pre-processing" Jul 30 03:11:13.361713 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.361602 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae8371ca-8291-46ff-8d5c-bef5e2679a8d" containerName="revoke-keys" Jul 30 03:11:13.361713 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.361611 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="08178535-f79e-4922-82e5-b3b8aa44a489" containerName="payload-pre-processing" Jul 30 03:11:13.361713 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.361619 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7457e3a-dc8e-44cf-8370-8c4ab07cedd9" containerName="payload-processing" Jul 30 03:11:13.364774 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.364754 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" Jul 30 03:11:13.368750 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.368731 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-da5459-dockercfg-p96cl\"" Jul 30 03:11:13.369038 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.368755 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-da5459\"" Jul 30 03:11:13.397129 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.397105 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx"] Jul 30 03:11:13.407001 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.406978 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8"] Jul 30 03:11:13.410286 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.410270 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" Jul 30 03:11:13.422759 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.422739 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8"] Jul 30 03:11:13.465118 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.465045 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f870d8d-754f-45fa-a558-504c58e33f1b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx\" (UID: \"7f870d8d-754f-45fa-a558-504c58e33f1b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" Jul 30 03:11:13.465236 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.465124 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpq2q\" (UniqueName: \"kubernetes.io/projected/7f870d8d-754f-45fa-a558-504c58e33f1b-kube-api-access-wpq2q\") pod \"payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx\" (UID: \"7f870d8d-754f-45fa-a558-504c58e33f1b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" Jul 30 03:11:13.565934 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.565905 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f870d8d-754f-45fa-a558-504c58e33f1b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx\" (UID: \"7f870d8d-754f-45fa-a558-504c58e33f1b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" Jul 30 03:11:13.566121 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.565945 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg96c\" (UniqueName: \"kubernetes.io/projected/02be289f-3540-42f6-b73a-b0f7de0c9ec5-kube-api-access-jg96c\") pod \"payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8\" (UID: \"02be289f-3540-42f6-b73a-b0f7de0c9ec5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" Jul 30 03:11:13.566121 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.565982 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02be289f-3540-42f6-b73a-b0f7de0c9ec5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8\" (UID: \"02be289f-3540-42f6-b73a-b0f7de0c9ec5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" Jul 30 03:11:13.566121 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.566083 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpq2q\" (UniqueName: \"kubernetes.io/projected/7f870d8d-754f-45fa-a558-504c58e33f1b-kube-api-access-wpq2q\") pod \"payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx\" (UID: \"7f870d8d-754f-45fa-a558-504c58e33f1b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" Jul 30 03:11:13.566611 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.566593 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f870d8d-754f-45fa-a558-504c58e33f1b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx\" (UID: \"7f870d8d-754f-45fa-a558-504c58e33f1b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" Jul 30 03:11:13.577321 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.577292 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpq2q\" (UniqueName: \"kubernetes.io/projected/7f870d8d-754f-45fa-a558-504c58e33f1b-kube-api-access-wpq2q\") pod \"payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx\" (UID: \"7f870d8d-754f-45fa-a558-504c58e33f1b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" Jul 30 03:11:13.667487 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.667458 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg96c\" (UniqueName: \"kubernetes.io/projected/02be289f-3540-42f6-b73a-b0f7de0c9ec5-kube-api-access-jg96c\") pod \"payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8\" (UID: \"02be289f-3540-42f6-b73a-b0f7de0c9ec5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" Jul 30 03:11:13.667623 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.667513 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02be289f-3540-42f6-b73a-b0f7de0c9ec5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8\" (UID: \"02be289f-3540-42f6-b73a-b0f7de0c9ec5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" Jul 30 03:11:13.668158 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.668136 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02be289f-3540-42f6-b73a-b0f7de0c9ec5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8\" (UID: \"02be289f-3540-42f6-b73a-b0f7de0c9ec5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" Jul 30 03:11:13.674281 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.674252 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" Jul 30 03:11:13.682898 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.682878 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg96c\" (UniqueName: \"kubernetes.io/projected/02be289f-3540-42f6-b73a-b0f7de0c9ec5-kube-api-access-jg96c\") pod \"payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8\" (UID: \"02be289f-3540-42f6-b73a-b0f7de0c9ec5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" Jul 30 03:11:13.719973 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.719910 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" Jul 30 03:11:13.817471 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.817434 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx"] Jul 30 03:11:13.820830 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:11:13.820792 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f870d8d_754f_45fa_a558_504c58e33f1b.slice/crio-5621b494db9fc17e250ec15f7981b27b3ab7c6acc559de65bc3d4bd4f0d041a8 WatchSource:0}: Error finding container 5621b494db9fc17e250ec15f7981b27b3ab7c6acc559de65bc3d4bd4f0d041a8: Status 404 returned error can't find the container with id 5621b494db9fc17e250ec15f7981b27b3ab7c6acc559de65bc3d4bd4f0d041a8 Jul 30 03:11:13.867589 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:13.867566 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8"] Jul 30 03:11:13.870700 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:11:13.870658 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02be289f_3540_42f6_b73a_b0f7de0c9ec5.slice/crio-5010d45a9e777c16ba279c713f118ad103d161fef5ccabf846f2dd85d08c7e2e WatchSource:0}: Error finding container 5010d45a9e777c16ba279c713f118ad103d161fef5ccabf846f2dd85d08c7e2e: Status 404 returned error can't find the container with id 5010d45a9e777c16ba279c713f118ad103d161fef5ccabf846f2dd85d08c7e2e Jul 30 03:11:14.572864 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:14.572827 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" event={"ID":"02be289f-3540-42f6-b73a-b0f7de0c9ec5","Type":"ContainerStarted","Data":"ca43c21e642ea1f8e0e97a285c0eb79ed62ffa68299c0dad5305b1fef76c2815"} Jul 30 03:11:14.573329 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:14.572947 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" Jul 30 03:11:14.573329 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:14.573016 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" event={"ID":"02be289f-3540-42f6-b73a-b0f7de0c9ec5","Type":"ContainerStarted","Data":"5010d45a9e777c16ba279c713f118ad103d161fef5ccabf846f2dd85d08c7e2e"} Jul 30 03:11:14.574304 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:14.574280 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" event={"ID":"7f870d8d-754f-45fa-a558-504c58e33f1b","Type":"ContainerStarted","Data":"6acba1e72f72f7889dcdbf0fd2a9ee181362896a436c7f5996c6bcc18762642f"} Jul 30 03:11:14.574430 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:14.574319 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" event={"ID":"7f870d8d-754f-45fa-a558-504c58e33f1b","Type":"ContainerStarted","Data":"5621b494db9fc17e250ec15f7981b27b3ab7c6acc559de65bc3d4bd4f0d041a8"} Jul 30 03:11:14.574430 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:14.574407 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" Jul 30 03:11:14.590703 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:14.590633 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" podStartSLOduration=1.5906198360000001 podStartE2EDuration="1.590619836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:11:14.588482103 +0000 UTC m=+2722.267286097" watchObservedRunningTime="2026-07-30 03:11:14.590619836 +0000 UTC m=+2722.269423870" Jul 30 03:11:14.606363 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:14.606318 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" podStartSLOduration=1.606304949 podStartE2EDuration="1.606304949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:11:14.606145988 +0000 UTC m=+2722.284949987" watchObservedRunningTime="2026-07-30 03:11:14.606304949 +0000 UTC m=+2722.285108946" Jul 30 03:11:16.720254 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:16.720212 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:17.050865 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:17.050833 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:17.363376 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:17.363280 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:18.094410 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:18.094377 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:20.354982 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:20.354947 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:20.521395 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:20.521358 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:20.729390 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:20.729305 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:20.829282 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:20.829247 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:21.347600 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.347562 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:21.447655 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.447613 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:21.569019 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.568985 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp"] Jul 30 03:11:21.572771 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.572745 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" Jul 30 03:11:21.577499 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.577480 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-f2a805-dockercfg-d24mt\"" Jul 30 03:11:21.577764 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.577749 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-f2a805\"" Jul 30 03:11:21.584414 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.584391 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp"] Jul 30 03:11:21.637337 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.637261 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc"] Jul 30 03:11:21.640695 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.640653 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" Jul 30 03:11:21.661111 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.661085 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc"] Jul 30 03:11:21.735491 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.735456 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh8mr\" (UniqueName: \"kubernetes.io/projected/351b6c0c-d9e5-4176-a1af-06b5c67e7347-kube-api-access-jh8mr\") pod \"payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp\" (UID: \"351b6c0c-d9e5-4176-a1af-06b5c67e7347\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" Jul 30 03:11:21.735688 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.735504 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9db2523f-0d42-42fb-9eef-f5110e708ac3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc\" (UID: \"9db2523f-0d42-42fb-9eef-f5110e708ac3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" Jul 30 03:11:21.735688 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.735615 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n9g5\" (UniqueName: \"kubernetes.io/projected/9db2523f-0d42-42fb-9eef-f5110e708ac3-kube-api-access-6n9g5\") pod \"payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc\" (UID: \"9db2523f-0d42-42fb-9eef-f5110e708ac3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" Jul 30 03:11:21.735688 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.735645 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351b6c0c-d9e5-4176-a1af-06b5c67e7347-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp\" (UID: \"351b6c0c-d9e5-4176-a1af-06b5c67e7347\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" Jul 30 03:11:21.836461 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.836415 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9db2523f-0d42-42fb-9eef-f5110e708ac3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc\" (UID: \"9db2523f-0d42-42fb-9eef-f5110e708ac3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" Jul 30 03:11:21.836642 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.836513 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n9g5\" (UniqueName: \"kubernetes.io/projected/9db2523f-0d42-42fb-9eef-f5110e708ac3-kube-api-access-6n9g5\") pod \"payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc\" (UID: \"9db2523f-0d42-42fb-9eef-f5110e708ac3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" Jul 30 03:11:21.836642 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.836544 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351b6c0c-d9e5-4176-a1af-06b5c67e7347-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp\" (UID: \"351b6c0c-d9e5-4176-a1af-06b5c67e7347\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" Jul 30 03:11:21.836642 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.836611 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh8mr\" (UniqueName: \"kubernetes.io/projected/351b6c0c-d9e5-4176-a1af-06b5c67e7347-kube-api-access-jh8mr\") pod \"payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp\" (UID: \"351b6c0c-d9e5-4176-a1af-06b5c67e7347\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" Jul 30 03:11:21.837197 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.837172 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9db2523f-0d42-42fb-9eef-f5110e708ac3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc\" (UID: \"9db2523f-0d42-42fb-9eef-f5110e708ac3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" Jul 30 03:11:21.837357 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.837336 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351b6c0c-d9e5-4176-a1af-06b5c67e7347-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp\" (UID: \"351b6c0c-d9e5-4176-a1af-06b5c67e7347\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" Jul 30 03:11:21.847192 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.847165 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n9g5\" (UniqueName: \"kubernetes.io/projected/9db2523f-0d42-42fb-9eef-f5110e708ac3-kube-api-access-6n9g5\") pod \"payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc\" (UID: \"9db2523f-0d42-42fb-9eef-f5110e708ac3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" Jul 30 03:11:21.847521 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.847504 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh8mr\" (UniqueName: \"kubernetes.io/projected/351b6c0c-d9e5-4176-a1af-06b5c67e7347-kube-api-access-jh8mr\") pod \"payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp\" (UID: \"351b6c0c-d9e5-4176-a1af-06b5c67e7347\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" Jul 30 03:11:21.883751 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.883725 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" Jul 30 03:11:21.952015 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:21.951990 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" Jul 30 03:11:22.024735 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:22.024540 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp"] Jul 30 03:11:22.028765 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:11:22.028731 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod351b6c0c_d9e5_4176_a1af_06b5c67e7347.slice/crio-8c2e1742b67ecfa2f42c1a9ad714ff0ce2149e6c4b441b172d2518de62ceea6e WatchSource:0}: Error finding container 8c2e1742b67ecfa2f42c1a9ad714ff0ce2149e6c4b441b172d2518de62ceea6e: Status 404 returned error can't find the container with id 8c2e1742b67ecfa2f42c1a9ad714ff0ce2149e6c4b441b172d2518de62ceea6e Jul 30 03:11:22.120321 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:22.120296 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc"] Jul 30 03:11:22.122225 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:11:22.122204 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9db2523f_0d42_42fb_9eef_f5110e708ac3.slice/crio-d166e2832724b6ae9a83c3a88cf3a158f6ba99195422838e7f790e3845753c27 WatchSource:0}: Error finding container d166e2832724b6ae9a83c3a88cf3a158f6ba99195422838e7f790e3845753c27: Status 404 returned error can't find the container with id d166e2832724b6ae9a83c3a88cf3a158f6ba99195422838e7f790e3845753c27 Jul 30 03:11:22.618317 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:22.618270 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" event={"ID":"351b6c0c-d9e5-4176-a1af-06b5c67e7347","Type":"ContainerStarted","Data":"3757737e5e6d809041182aa849bba88a9cf5d4c71a0a8ead8c9cfa363a59ae98"} Jul 30 03:11:22.618793 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:22.618341 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" Jul 30 03:11:22.618793 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:22.618364 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" event={"ID":"351b6c0c-d9e5-4176-a1af-06b5c67e7347","Type":"ContainerStarted","Data":"8c2e1742b67ecfa2f42c1a9ad714ff0ce2149e6c4b441b172d2518de62ceea6e"} Jul 30 03:11:22.619940 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:22.619916 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" event={"ID":"9db2523f-0d42-42fb-9eef-f5110e708ac3","Type":"ContainerStarted","Data":"8f2b09551cf8025f830aa906757286ea00d2487504af6d63730ffa087a2d131d"} Jul 30 03:11:22.620067 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:22.619965 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" Jul 30 03:11:22.620067 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:22.619989 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" event={"ID":"9db2523f-0d42-42fb-9eef-f5110e708ac3","Type":"ContainerStarted","Data":"d166e2832724b6ae9a83c3a88cf3a158f6ba99195422838e7f790e3845753c27"} Jul 30 03:11:22.635724 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:22.635652 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" podStartSLOduration=1.635635571 podStartE2EDuration="1.635635571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:11:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:11:22.635367588 +0000 UTC m=+2730.314171586" watchObservedRunningTime="2026-07-30 03:11:22.635635571 +0000 UTC m=+2730.314439568" Jul 30 03:11:22.654425 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:22.654381 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" podStartSLOduration=1.6543678370000001 podStartE2EDuration="1.654367837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:11:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:11:22.653730547 +0000 UTC m=+2730.332534541" watchObservedRunningTime="2026-07-30 03:11:22.654367837 +0000 UTC m=+2730.333171832" Jul 30 03:11:24.714087 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:24.714041 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:11:25.581854 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:25.581819 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" Jul 30 03:11:25.582028 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:25.581932 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" Jul 30 03:11:33.627273 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:33.627231 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" Jul 30 03:11:33.627744 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:33.627324 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" Jul 30 03:11:34.871741 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:11:34.871710 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:12:06.570832 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:12:06.570789 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:14:58.623362 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:14:58.623314 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:15:00.029108 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.029078 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:15:00.132975 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.132939 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-da5459-29756355-dnpmw"] Jul 30 03:15:00.136759 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.136738 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756355-jthnt"] Jul 30 03:15:00.136904 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.136865 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-da5459-29756355-dnpmw" Jul 30 03:15:00.139104 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.139083 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2npk7\"" Jul 30 03:15:00.140464 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.140445 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756355-jthnt" Jul 30 03:15:00.149480 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.148383 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-da5459-29756355-dnpmw" Jul 30 03:15:00.157803 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.157747 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-da5459-29756355-dnpmw"] Jul 30 03:15:00.158323 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.158300 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756355-jthnt"] Jul 30 03:15:00.161768 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.161752 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756355-jthnt" Jul 30 03:15:00.312325 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.312295 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-da5459-29756355-dnpmw"] Jul 30 03:15:00.314173 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:15:00.314119 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d111f3c_380b_483b_a585_a2b97b48eaf2.slice/crio-c7a56b14c388b34a4dddf27e6804f7da910542069870c9d55a036f546a12ecf3 WatchSource:0}: Error finding container c7a56b14c388b34a4dddf27e6804f7da910542069870c9d55a036f546a12ecf3: Status 404 returned error can't find the container with id c7a56b14c388b34a4dddf27e6804f7da910542069870c9d55a036f546a12ecf3 Jul 30 03:15:00.342339 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.342311 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756355-jthnt"] Jul 30 03:15:00.343265 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:15:00.343241 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32d69bd9_c3f0_4c6c_8d69_19de3c31dabe.slice/crio-f5bb92454b6eb45631a064c7771e3eb768833ad8379067c804be5d2994a69049 WatchSource:0}: Error finding container f5bb92454b6eb45631a064c7771e3eb768833ad8379067c804be5d2994a69049: Status 404 returned error can't find the container with id f5bb92454b6eb45631a064c7771e3eb768833ad8379067c804be5d2994a69049 Jul 30 03:15:00.677133 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.677102 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-da5459-29756355-dnpmw" event={"ID":"5d111f3c-380b-483b-a585-a2b97b48eaf2","Type":"ContainerStarted","Data":"c7a56b14c388b34a4dddf27e6804f7da910542069870c9d55a036f546a12ecf3"} Jul 30 03:15:00.678724 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.678696 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756355-jthnt" event={"ID":"32d69bd9-c3f0-4c6c-8d69-19de3c31dabe","Type":"ContainerStarted","Data":"1693a9519876280d36061b02e3634eff8881f7d4fe90bf9cf3039b51f3d8dcff"} Jul 30 03:15:00.678862 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.678750 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756355-jthnt" event={"ID":"32d69bd9-c3f0-4c6c-8d69-19de3c31dabe","Type":"ContainerStarted","Data":"f5bb92454b6eb45631a064c7771e3eb768833ad8379067c804be5d2994a69049"} Jul 30 03:15:00.693174 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:00.693127 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756355-jthnt" podStartSLOduration=0.693109716 podStartE2EDuration="693.109716ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:15:00.691905918 +0000 UTC m=+2948.370709926" watchObservedRunningTime="2026-07-30 03:15:00.693109716 +0000 UTC m=+2948.371913711" Jul 30 03:15:01.684533 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:01.684491 2566 generic.go:356] "Generic (PLEG): container finished" podID="32d69bd9-c3f0-4c6c-8d69-19de3c31dabe" containerID="1693a9519876280d36061b02e3634eff8881f7d4fe90bf9cf3039b51f3d8dcff" exitCode=0 Jul 30 03:15:01.684975 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:01.684571 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756355-jthnt" event={"ID":"32d69bd9-c3f0-4c6c-8d69-19de3c31dabe","Type":"ContainerDied","Data":"1693a9519876280d36061b02e3634eff8881f7d4fe90bf9cf3039b51f3d8dcff"} Jul 30 03:15:01.686111 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:01.686064 2566 generic.go:356] "Generic (PLEG): container finished" podID="5d111f3c-380b-483b-a585-a2b97b48eaf2" containerID="615ead914a4191732ad6012368c7b940efaa15863cb74406a53f95e6845708ce" exitCode=0 Jul 30 03:15:01.686111 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:01.686103 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-da5459-29756355-dnpmw" event={"ID":"5d111f3c-380b-483b-a585-a2b97b48eaf2","Type":"ContainerDied","Data":"615ead914a4191732ad6012368c7b940efaa15863cb74406a53f95e6845708ce"} Jul 30 03:15:02.839912 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:02.839885 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756355-jthnt" Jul 30 03:15:02.871539 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:02.871516 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-da5459-29756355-dnpmw" Jul 30 03:15:03.696979 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:03.696952 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-da5459-29756355-dnpmw" Jul 30 03:15:03.696979 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:03.696967 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-da5459-29756355-dnpmw" event={"ID":"5d111f3c-380b-483b-a585-a2b97b48eaf2","Type":"ContainerDied","Data":"c7a56b14c388b34a4dddf27e6804f7da910542069870c9d55a036f546a12ecf3"} Jul 30 03:15:03.697243 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:03.697014 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7a56b14c388b34a4dddf27e6804f7da910542069870c9d55a036f546a12ecf3" Jul 30 03:15:03.698222 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:03.698202 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756355-jthnt" event={"ID":"32d69bd9-c3f0-4c6c-8d69-19de3c31dabe","Type":"ContainerDied","Data":"f5bb92454b6eb45631a064c7771e3eb768833ad8379067c804be5d2994a69049"} Jul 30 03:15:03.698315 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:03.698236 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756355-jthnt" Jul 30 03:15:03.698354 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:03.698237 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5bb92454b6eb45631a064c7771e3eb768833ad8379067c804be5d2994a69049" Jul 30 03:15:43.390696 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:43.390645 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:15:53.071807 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:53.071777 2566 scope.go:117] "RemoveContainer" containerID="cb71f88b32aa481e762e66a50b2d8df78f47cdb595b98f970a9d70bc938fa826" Jul 30 03:15:53.080165 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:15:53.080144 2566 scope.go:117] "RemoveContainer" containerID="64896233ae7748d8ba33809455ffef941e95e07b42e3f1f84becaa2b3c2f9238" Jul 30 03:16:23.018485 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:16:23.018449 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:16:37.846267 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:16:37.846229 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:16:38.142366 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:16:38.142286 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:16:39.049091 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:16:39.049054 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:16:53.101851 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:16:53.101821 2566 scope.go:117] "RemoveContainer" containerID="2717edb03912d4dd6aa89e6ed21cd49a4e735d97f14ed8965cd813c09f47cf26" Jul 30 03:16:53.109810 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:16:53.109788 2566 scope.go:117] "RemoveContainer" containerID="66032ead3fbb5b51252c8599c9c1a5aa122cd30f931891c42b9d84b9e44fc554" Jul 30 03:17:02.032338 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:17:02.032303 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:17:29.650156 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:17:29.650123 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:18:34.102098 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:18:34.102066 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:18:35.046922 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:18:35.046887 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:18:40.521016 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:18:40.520976 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:18:49.658054 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:18:49.658018 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:19:11.960606 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:19:11.960575 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:03.247722 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:03.247688 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:15.022440 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:15.022403 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:16.057922 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:16.057882 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:34.986211 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:34.986178 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:35.309189 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:35.309147 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:36.032627 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:36.032584 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:38.516153 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:38.516119 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:38.606734 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:38.606706 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:38.680812 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:38.680768 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:38.975114 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:38.975032 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:39.183291 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.183263 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:39.278197 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.278169 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:39.288116 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.288085 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m"] Jul 30 03:20:39.288483 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.288471 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32d69bd9-c3f0-4c6c-8d69-19de3c31dabe" containerName="cleanup" Jul 30 03:20:39.288528 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.288485 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="32d69bd9-c3f0-4c6c-8d69-19de3c31dabe" containerName="cleanup" Jul 30 03:20:39.288528 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.288508 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d111f3c-380b-483b-a585-a2b97b48eaf2" containerName="cleanup" Jul 30 03:20:39.288528 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.288514 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d111f3c-380b-483b-a585-a2b97b48eaf2" containerName="cleanup" Jul 30 03:20:39.288618 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.288566 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="32d69bd9-c3f0-4c6c-8d69-19de3c31dabe" containerName="cleanup" Jul 30 03:20:39.288618 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.288576 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d111f3c-380b-483b-a585-a2b97b48eaf2" containerName="cleanup" Jul 30 03:20:39.291529 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.291514 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" Jul 30 03:20:39.294077 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.294055 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-2cad57-dockercfg-zr65t\"" Jul 30 03:20:39.294203 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.294079 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-2cad57\"" Jul 30 03:20:39.303561 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.303538 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m"] Jul 30 03:20:39.313884 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.313857 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c"] Jul 30 03:20:39.317487 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.317470 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" Jul 30 03:20:39.326136 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.326107 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c"] Jul 30 03:20:39.410258 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.410222 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvf29\" (UniqueName: \"kubernetes.io/projected/b9e822c9-ba63-4f35-996a-b593edea73b1-kube-api-access-cvf29\") pod \"payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c\" (UID: \"b9e822c9-ba63-4f35-996a-b593edea73b1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" Jul 30 03:20:39.410435 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.410343 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eae1883-5596-4335-aa03-d7e231ba8513-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m\" (UID: \"8eae1883-5596-4335-aa03-d7e231ba8513\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" Jul 30 03:20:39.410435 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.410393 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9e822c9-ba63-4f35-996a-b593edea73b1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c\" (UID: \"b9e822c9-ba63-4f35-996a-b593edea73b1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" Jul 30 03:20:39.410435 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.410415 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r88pq\" (UniqueName: \"kubernetes.io/projected/8eae1883-5596-4335-aa03-d7e231ba8513-kube-api-access-r88pq\") pod \"payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m\" (UID: \"8eae1883-5596-4335-aa03-d7e231ba8513\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" Jul 30 03:20:39.511824 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.511780 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvf29\" (UniqueName: \"kubernetes.io/projected/b9e822c9-ba63-4f35-996a-b593edea73b1-kube-api-access-cvf29\") pod \"payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c\" (UID: \"b9e822c9-ba63-4f35-996a-b593edea73b1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" Jul 30 03:20:39.512026 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.511882 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eae1883-5596-4335-aa03-d7e231ba8513-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m\" (UID: \"8eae1883-5596-4335-aa03-d7e231ba8513\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" Jul 30 03:20:39.512026 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.511929 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9e822c9-ba63-4f35-996a-b593edea73b1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c\" (UID: \"b9e822c9-ba63-4f35-996a-b593edea73b1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" Jul 30 03:20:39.512026 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.511951 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r88pq\" (UniqueName: \"kubernetes.io/projected/8eae1883-5596-4335-aa03-d7e231ba8513-kube-api-access-r88pq\") pod \"payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m\" (UID: \"8eae1883-5596-4335-aa03-d7e231ba8513\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" Jul 30 03:20:39.512611 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.512541 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9e822c9-ba63-4f35-996a-b593edea73b1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c\" (UID: \"b9e822c9-ba63-4f35-996a-b593edea73b1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" Jul 30 03:20:39.512611 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.512598 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eae1883-5596-4335-aa03-d7e231ba8513-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m\" (UID: \"8eae1883-5596-4335-aa03-d7e231ba8513\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" Jul 30 03:20:39.520118 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.520095 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvf29\" (UniqueName: \"kubernetes.io/projected/b9e822c9-ba63-4f35-996a-b593edea73b1-kube-api-access-cvf29\") pod \"payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c\" (UID: \"b9e822c9-ba63-4f35-996a-b593edea73b1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" Jul 30 03:20:39.520722 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.520704 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r88pq\" (UniqueName: \"kubernetes.io/projected/8eae1883-5596-4335-aa03-d7e231ba8513-kube-api-access-r88pq\") pod \"payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m\" (UID: \"8eae1883-5596-4335-aa03-d7e231ba8513\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" Jul 30 03:20:39.602267 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.602201 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" Jul 30 03:20:39.628396 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.628371 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" Jul 30 03:20:39.949780 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.949753 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m"] Jul 30 03:20:39.951371 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:20:39.951334 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eae1883_5596_4335_aa03_d7e231ba8513.slice/crio-be94b44747ff587cf5e10db285ab121e11c37302349e4512499c9d8500ef3d7f WatchSource:0}: Error finding container be94b44747ff587cf5e10db285ab121e11c37302349e4512499c9d8500ef3d7f: Status 404 returned error can't find the container with id be94b44747ff587cf5e10db285ab121e11c37302349e4512499c9d8500ef3d7f Jul 30 03:20:39.976699 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:39.976674 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c"] Jul 30 03:20:39.981737 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:20:39.981713 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9e822c9_ba63_4f35_996a_b593edea73b1.slice/crio-128b716d15e4fe7e283ad07895d4836051f9b037342f7dd4dba94425d54ced3f WatchSource:0}: Error finding container 128b716d15e4fe7e283ad07895d4836051f9b037342f7dd4dba94425d54ced3f: Status 404 returned error can't find the container with id 128b716d15e4fe7e283ad07895d4836051f9b037342f7dd4dba94425d54ced3f Jul 30 03:20:40.305329 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:40.305293 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" event={"ID":"8eae1883-5596-4335-aa03-d7e231ba8513","Type":"ContainerStarted","Data":"caf2550b8c031e31437cc40e3a92d5de452acd0c8935c67d8bccc2dd690ba901"} Jul 30 03:20:40.305496 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:40.305350 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" event={"ID":"8eae1883-5596-4335-aa03-d7e231ba8513","Type":"ContainerStarted","Data":"be94b44747ff587cf5e10db285ab121e11c37302349e4512499c9d8500ef3d7f"} Jul 30 03:20:40.305496 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:40.305458 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" Jul 30 03:20:40.306815 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:40.306793 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" event={"ID":"b9e822c9-ba63-4f35-996a-b593edea73b1","Type":"ContainerStarted","Data":"7d0597e72f2500f4e4d4f562d40c5d237a9b6e8a344e5aa73968ba2882c7b1c2"} Jul 30 03:20:40.306917 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:40.306831 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" event={"ID":"b9e822c9-ba63-4f35-996a-b593edea73b1","Type":"ContainerStarted","Data":"128b716d15e4fe7e283ad07895d4836051f9b037342f7dd4dba94425d54ced3f"} Jul 30 03:20:40.306917 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:40.306859 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" Jul 30 03:20:40.324262 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:40.324224 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" podStartSLOduration=1.324208299 podStartE2EDuration="1.324208299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:20:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:20:40.322100738 +0000 UTC m=+3288.000904731" watchObservedRunningTime="2026-07-30 03:20:40.324208299 +0000 UTC m=+3288.003012294" Jul 30 03:20:40.341553 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:40.341509 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" podStartSLOduration=1.3414912700000001 podStartE2EDuration="1.34149127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:20:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:20:40.339657718 +0000 UTC m=+3288.018461712" watchObservedRunningTime="2026-07-30 03:20:40.34149127 +0000 UTC m=+3288.020295268" Jul 30 03:20:43.009807 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:43.009777 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:51.314353 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:51.314319 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" Jul 30 03:20:51.314761 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:51.314400 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" Jul 30 03:20:53.952568 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:53.952533 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:54.279469 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:54.279426 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:54.384205 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:54.384176 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:57.773918 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:57.773885 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:57.874994 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:57.874958 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:57.943851 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:57.943808 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:58.378764 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.378734 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:58.495716 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.495686 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:58.596299 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.596237 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:20:58.635844 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.635780 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv"] Jul 30 03:20:58.640736 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.640718 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" Jul 30 03:20:58.643586 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.643559 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-f58ab6-dockercfg-smxkx\"" Jul 30 03:20:58.643721 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.643571 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-f58ab6\"" Jul 30 03:20:58.650244 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.650224 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv"] Jul 30 03:20:58.677107 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.677081 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/accfe378-7122-4d32-9b87-ff9718371bce-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv\" (UID: \"accfe378-7122-4d32-9b87-ff9718371bce\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" Jul 30 03:20:58.677195 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.677119 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58sml\" (UniqueName: \"kubernetes.io/projected/accfe378-7122-4d32-9b87-ff9718371bce-kube-api-access-58sml\") pod \"payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv\" (UID: \"accfe378-7122-4d32-9b87-ff9718371bce\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" Jul 30 03:20:58.699570 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.699547 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5"] Jul 30 03:20:58.703004 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.702990 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" Jul 30 03:20:58.712976 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.712956 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5"] Jul 30 03:20:58.778137 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.778094 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c220c6bd-7c2c-4077-8783-30e1c1c60653-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5\" (UID: \"c220c6bd-7c2c-4077-8783-30e1c1c60653\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" Jul 30 03:20:58.778601 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.778198 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/accfe378-7122-4d32-9b87-ff9718371bce-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv\" (UID: \"accfe378-7122-4d32-9b87-ff9718371bce\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" Jul 30 03:20:58.778601 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.778246 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58sml\" (UniqueName: \"kubernetes.io/projected/accfe378-7122-4d32-9b87-ff9718371bce-kube-api-access-58sml\") pod \"payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv\" (UID: \"accfe378-7122-4d32-9b87-ff9718371bce\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" Jul 30 03:20:58.778601 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.778287 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xrvx\" (UniqueName: \"kubernetes.io/projected/c220c6bd-7c2c-4077-8783-30e1c1c60653-kube-api-access-4xrvx\") pod \"payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5\" (UID: \"c220c6bd-7c2c-4077-8783-30e1c1c60653\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" Jul 30 03:20:58.779048 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.779024 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/accfe378-7122-4d32-9b87-ff9718371bce-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv\" (UID: \"accfe378-7122-4d32-9b87-ff9718371bce\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" Jul 30 03:20:58.786723 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.786705 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58sml\" (UniqueName: \"kubernetes.io/projected/accfe378-7122-4d32-9b87-ff9718371bce-kube-api-access-58sml\") pod \"payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv\" (UID: \"accfe378-7122-4d32-9b87-ff9718371bce\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" Jul 30 03:20:58.879624 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.879583 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c220c6bd-7c2c-4077-8783-30e1c1c60653-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5\" (UID: \"c220c6bd-7c2c-4077-8783-30e1c1c60653\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" Jul 30 03:20:58.879781 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.879757 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xrvx\" (UniqueName: \"kubernetes.io/projected/c220c6bd-7c2c-4077-8783-30e1c1c60653-kube-api-access-4xrvx\") pod \"payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5\" (UID: \"c220c6bd-7c2c-4077-8783-30e1c1c60653\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" Jul 30 03:20:58.881497 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.881149 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c220c6bd-7c2c-4077-8783-30e1c1c60653-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5\" (UID: \"c220c6bd-7c2c-4077-8783-30e1c1c60653\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" Jul 30 03:20:58.890455 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.890404 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xrvx\" (UniqueName: \"kubernetes.io/projected/c220c6bd-7c2c-4077-8783-30e1c1c60653-kube-api-access-4xrvx\") pod \"payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5\" (UID: \"c220c6bd-7c2c-4077-8783-30e1c1c60653\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" Jul 30 03:20:58.952235 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:58.952213 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" Jul 30 03:20:59.012916 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:59.012884 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" Jul 30 03:20:59.093206 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:59.093180 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv"] Jul 30 03:20:59.096089 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:20:59.096063 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaccfe378_7122_4d32_9b87_ff9718371bce.slice/crio-53975abbcbc505b206e39048dc7a80e416dc4b024d87b9167ad105093a611060 WatchSource:0}: Error finding container 53975abbcbc505b206e39048dc7a80e416dc4b024d87b9167ad105093a611060: Status 404 returned error can't find the container with id 53975abbcbc505b206e39048dc7a80e416dc4b024d87b9167ad105093a611060 Jul 30 03:20:59.156441 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:59.156418 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5"] Jul 30 03:20:59.158974 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:20:59.158942 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc220c6bd_7c2c_4077_8783_30e1c1c60653.slice/crio-3c7cb1f1629005d37da1ecafb2c2f3099f4fb860c0fa6dcfaf4bfd1667bc7e5e WatchSource:0}: Error finding container 3c7cb1f1629005d37da1ecafb2c2f3099f4fb860c0fa6dcfaf4bfd1667bc7e5e: Status 404 returned error can't find the container with id 3c7cb1f1629005d37da1ecafb2c2f3099f4fb860c0fa6dcfaf4bfd1667bc7e5e Jul 30 03:20:59.401930 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:59.401847 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" event={"ID":"c220c6bd-7c2c-4077-8783-30e1c1c60653","Type":"ContainerStarted","Data":"06aaf1459fcff5ae2c4fb7affa8cb3ef0a4a8ac2651a3a863a8c9e8a2e3ef40c"} Jul 30 03:20:59.401930 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:59.401899 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" event={"ID":"c220c6bd-7c2c-4077-8783-30e1c1c60653","Type":"ContainerStarted","Data":"3c7cb1f1629005d37da1ecafb2c2f3099f4fb860c0fa6dcfaf4bfd1667bc7e5e"} Jul 30 03:20:59.402181 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:59.401954 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" Jul 30 03:20:59.403441 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:59.403416 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" event={"ID":"accfe378-7122-4d32-9b87-ff9718371bce","Type":"ContainerStarted","Data":"14905c0cf0dbef8aa580883c0cdce081e78f0eff68cc0d4dd6d00a004a7e692b"} Jul 30 03:20:59.403543 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:59.403467 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" event={"ID":"accfe378-7122-4d32-9b87-ff9718371bce","Type":"ContainerStarted","Data":"53975abbcbc505b206e39048dc7a80e416dc4b024d87b9167ad105093a611060"} Jul 30 03:20:59.403543 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:59.403496 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" Jul 30 03:20:59.436714 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:59.436641 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" podStartSLOduration=1.436624754 podStartE2EDuration="1.436624754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:20:59.436030043 +0000 UTC m=+3307.114834061" watchObservedRunningTime="2026-07-30 03:20:59.436624754 +0000 UTC m=+3307.115428750" Jul 30 03:20:59.480787 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:20:59.480743 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" podStartSLOduration=1.480729869 podStartE2EDuration="1.480729869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:20:59.478250223 +0000 UTC m=+3307.157054219" watchObservedRunningTime="2026-07-30 03:20:59.480729869 +0000 UTC m=+3307.159533865" Jul 30 03:21:02.179061 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:21:02.179023 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:21:05.989102 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:21:05.989067 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:21:10.410754 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:21:10.410711 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" podUID="c220c6bd-7c2c-4077-8783-30e1c1c60653" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.55:9004: connect: connection refused" Jul 30 03:21:10.411193 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:21:10.411059 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" Jul 30 03:21:20.410058 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:21:20.410026 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" Jul 30 03:21:29.593838 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:21:29.593807 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv_accfe378-7122-4d32-9b87-ff9718371bce/payload-processing/0.log" Jul 30 03:21:38.272313 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:21:38.272271 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:24.532393 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:24.532364 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c_b9e822c9-ba63-4f35-996a-b593edea73b1/payload-processing/0.log" Jul 30 03:22:24.777194 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:24.777164 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv_accfe378-7122-4d32-9b87-ff9718371bce/payload-processing/0.log" Jul 30 03:22:25.044909 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:25.044865 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:25.473457 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:25.473350 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:25.559838 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:25.559811 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:28.267670 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:28.267624 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:28.364587 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:28.364554 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:28.448726 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:28.448695 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:28.558833 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:28.558751 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:28.935218 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:28.935149 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:29.016382 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.016353 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:29.041439 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.041411 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb"] Jul 30 03:22:29.045032 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.045016 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" Jul 30 03:22:29.047617 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.047600 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-64f2f8-dockercfg-str7v\"" Jul 30 03:22:29.047703 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.047620 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-64f2f8\"" Jul 30 03:22:29.055064 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.055043 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb"] Jul 30 03:22:29.074995 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.074965 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x"] Jul 30 03:22:29.078558 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.078539 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" Jul 30 03:22:29.087006 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.086987 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x"] Jul 30 03:22:29.164708 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.164649 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/423a59cf-a528-402c-97f9-4783e708ee9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb\" (UID: \"423a59cf-a528-402c-97f9-4783e708ee9c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" Jul 30 03:22:29.164708 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.164716 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d49s9\" (UniqueName: \"kubernetes.io/projected/423a59cf-a528-402c-97f9-4783e708ee9c-kube-api-access-d49s9\") pod \"payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb\" (UID: \"423a59cf-a528-402c-97f9-4783e708ee9c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" Jul 30 03:22:29.164708 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.164738 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/555c2dc9-cd37-4e45-be48-1b66617e259e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x\" (UID: \"555c2dc9-cd37-4e45-be48-1b66617e259e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" Jul 30 03:22:29.165044 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.164783 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wktsj\" (UniqueName: \"kubernetes.io/projected/555c2dc9-cd37-4e45-be48-1b66617e259e-kube-api-access-wktsj\") pod \"payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x\" (UID: \"555c2dc9-cd37-4e45-be48-1b66617e259e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" Jul 30 03:22:29.266182 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.266149 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wktsj\" (UniqueName: \"kubernetes.io/projected/555c2dc9-cd37-4e45-be48-1b66617e259e-kube-api-access-wktsj\") pod \"payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x\" (UID: \"555c2dc9-cd37-4e45-be48-1b66617e259e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" Jul 30 03:22:29.266329 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.266258 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/423a59cf-a528-402c-97f9-4783e708ee9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb\" (UID: \"423a59cf-a528-402c-97f9-4783e708ee9c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" Jul 30 03:22:29.266329 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.266275 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d49s9\" (UniqueName: \"kubernetes.io/projected/423a59cf-a528-402c-97f9-4783e708ee9c-kube-api-access-d49s9\") pod \"payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb\" (UID: \"423a59cf-a528-402c-97f9-4783e708ee9c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" Jul 30 03:22:29.266329 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.266290 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/555c2dc9-cd37-4e45-be48-1b66617e259e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x\" (UID: \"555c2dc9-cd37-4e45-be48-1b66617e259e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" Jul 30 03:22:29.266963 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.266945 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/555c2dc9-cd37-4e45-be48-1b66617e259e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x\" (UID: \"555c2dc9-cd37-4e45-be48-1b66617e259e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" Jul 30 03:22:29.267009 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.266991 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/423a59cf-a528-402c-97f9-4783e708ee9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb\" (UID: \"423a59cf-a528-402c-97f9-4783e708ee9c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" Jul 30 03:22:29.273817 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.273788 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d49s9\" (UniqueName: \"kubernetes.io/projected/423a59cf-a528-402c-97f9-4783e708ee9c-kube-api-access-d49s9\") pod \"payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb\" (UID: \"423a59cf-a528-402c-97f9-4783e708ee9c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" Jul 30 03:22:29.274125 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.273839 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wktsj\" (UniqueName: \"kubernetes.io/projected/555c2dc9-cd37-4e45-be48-1b66617e259e-kube-api-access-wktsj\") pod \"payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x\" (UID: \"555c2dc9-cd37-4e45-be48-1b66617e259e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" Jul 30 03:22:29.357374 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.357347 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" Jul 30 03:22:29.389527 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.389502 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" Jul 30 03:22:29.512819 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.512768 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb"] Jul 30 03:22:29.515725 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:22:29.515686 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod423a59cf_a528_402c_97f9_4783e708ee9c.slice/crio-91df3cd7396bce8f528136f2fa386edfdedca68eb35d5f543be4cd3ce0a7eaf7 WatchSource:0}: Error finding container 91df3cd7396bce8f528136f2fa386edfdedca68eb35d5f543be4cd3ce0a7eaf7: Status 404 returned error can't find the container with id 91df3cd7396bce8f528136f2fa386edfdedca68eb35d5f543be4cd3ce0a7eaf7 Jul 30 03:22:29.754278 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.754252 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x"] Jul 30 03:22:29.755655 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:22:29.755622 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod555c2dc9_cd37_4e45_be48_1b66617e259e.slice/crio-ebf0822340e7bf40837772af6ecdb4b30b91fc3a44da51b5c7a07bc8ec08d42a WatchSource:0}: Error finding container ebf0822340e7bf40837772af6ecdb4b30b91fc3a44da51b5c7a07bc8ec08d42a: Status 404 returned error can't find the container with id ebf0822340e7bf40837772af6ecdb4b30b91fc3a44da51b5c7a07bc8ec08d42a Jul 30 03:22:29.857342 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.857306 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" event={"ID":"555c2dc9-cd37-4e45-be48-1b66617e259e","Type":"ContainerStarted","Data":"ad861a51acb0baa874dbd7238409301d609638ac8a31c29959dc0dfad3c4b8bc"} Jul 30 03:22:29.857520 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.857365 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" event={"ID":"555c2dc9-cd37-4e45-be48-1b66617e259e","Type":"ContainerStarted","Data":"ebf0822340e7bf40837772af6ecdb4b30b91fc3a44da51b5c7a07bc8ec08d42a"} Jul 30 03:22:29.857520 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.857394 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" Jul 30 03:22:29.858898 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.858867 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" event={"ID":"423a59cf-a528-402c-97f9-4783e708ee9c","Type":"ContainerStarted","Data":"b2042fece18785b35b90728b813ec98f76735ae0c23867d7737c1ddfd90cd8c4"} Jul 30 03:22:29.859020 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.858910 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" event={"ID":"423a59cf-a528-402c-97f9-4783e708ee9c","Type":"ContainerStarted","Data":"91df3cd7396bce8f528136f2fa386edfdedca68eb35d5f543be4cd3ce0a7eaf7"} Jul 30 03:22:29.859020 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.858933 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" Jul 30 03:22:29.874734 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.874690 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" podStartSLOduration=0.874656047 podStartE2EDuration="874.656047ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:22:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:22:29.873071487 +0000 UTC m=+3397.551875485" watchObservedRunningTime="2026-07-30 03:22:29.874656047 +0000 UTC m=+3397.553460042" Jul 30 03:22:29.889111 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:29.889060 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" podStartSLOduration=0.889045554 podStartE2EDuration="889.045554ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 03:22:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:22:29.887313651 +0000 UTC m=+3397.566117652" watchObservedRunningTime="2026-07-30 03:22:29.889045554 +0000 UTC m=+3397.567849619" Jul 30 03:22:32.776445 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:32.776410 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:40.865153 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:40.865127 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" Jul 30 03:22:40.865558 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:40.865185 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" Jul 30 03:22:54.059283 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.059254 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:54.084439 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.084407 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x"] Jul 30 03:22:54.084730 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.084701 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" podUID="555c2dc9-cd37-4e45-be48-1b66617e259e" containerName="payload-processing" containerID="cri-o://ad861a51acb0baa874dbd7238409301d609638ac8a31c29959dc0dfad3c4b8bc" gracePeriod=30 Jul 30 03:22:54.090899 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.090844 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb"] Jul 30 03:22:54.091137 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.091072 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" podUID="423a59cf-a528-402c-97f9-4783e708ee9c" containerName="payload-pre-processing" containerID="cri-o://b2042fece18785b35b90728b813ec98f76735ae0c23867d7737c1ddfd90cd8c4" gracePeriod=30 Jul 30 03:22:54.164138 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.164105 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:54.263837 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.263803 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:54.377528 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.377505 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" Jul 30 03:22:54.379711 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.379693 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" Jul 30 03:22:54.506047 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.506015 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/555c2dc9-cd37-4e45-be48-1b66617e259e-plugins-config-volume\") pod \"555c2dc9-cd37-4e45-be48-1b66617e259e\" (UID: \"555c2dc9-cd37-4e45-be48-1b66617e259e\") " Jul 30 03:22:54.506209 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.506059 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d49s9\" (UniqueName: \"kubernetes.io/projected/423a59cf-a528-402c-97f9-4783e708ee9c-kube-api-access-d49s9\") pod \"423a59cf-a528-402c-97f9-4783e708ee9c\" (UID: \"423a59cf-a528-402c-97f9-4783e708ee9c\") " Jul 30 03:22:54.506209 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.506197 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wktsj\" (UniqueName: \"kubernetes.io/projected/555c2dc9-cd37-4e45-be48-1b66617e259e-kube-api-access-wktsj\") pod \"555c2dc9-cd37-4e45-be48-1b66617e259e\" (UID: \"555c2dc9-cd37-4e45-be48-1b66617e259e\") " Jul 30 03:22:54.506285 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.506232 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/423a59cf-a528-402c-97f9-4783e708ee9c-plugins-config-volume\") pod \"423a59cf-a528-402c-97f9-4783e708ee9c\" (UID: \"423a59cf-a528-402c-97f9-4783e708ee9c\") " Jul 30 03:22:54.506367 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.506339 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/555c2dc9-cd37-4e45-be48-1b66617e259e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "555c2dc9-cd37-4e45-be48-1b66617e259e" (UID: "555c2dc9-cd37-4e45-be48-1b66617e259e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:22:54.506585 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.506562 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/555c2dc9-cd37-4e45-be48-1b66617e259e-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:22:54.506703 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.506618 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/423a59cf-a528-402c-97f9-4783e708ee9c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "423a59cf-a528-402c-97f9-4783e708ee9c" (UID: "423a59cf-a528-402c-97f9-4783e708ee9c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:22:54.508111 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.508091 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/423a59cf-a528-402c-97f9-4783e708ee9c-kube-api-access-d49s9" (OuterVolumeSpecName: "kube-api-access-d49s9") pod "423a59cf-a528-402c-97f9-4783e708ee9c" (UID: "423a59cf-a528-402c-97f9-4783e708ee9c"). InnerVolumeSpecName "kube-api-access-d49s9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:22:54.508200 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.508153 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/555c2dc9-cd37-4e45-be48-1b66617e259e-kube-api-access-wktsj" (OuterVolumeSpecName: "kube-api-access-wktsj") pod "555c2dc9-cd37-4e45-be48-1b66617e259e" (UID: "555c2dc9-cd37-4e45-be48-1b66617e259e"). InnerVolumeSpecName "kube-api-access-wktsj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:22:54.607800 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.607707 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d49s9\" (UniqueName: \"kubernetes.io/projected/423a59cf-a528-402c-97f9-4783e708ee9c-kube-api-access-d49s9\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:22:54.607800 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.607749 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wktsj\" (UniqueName: \"kubernetes.io/projected/555c2dc9-cd37-4e45-be48-1b66617e259e-kube-api-access-wktsj\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:22:54.607800 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.607759 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/423a59cf-a528-402c-97f9-4783e708ee9c-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:22:54.982057 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.982024 2566 generic.go:356] "Generic (PLEG): container finished" podID="555c2dc9-cd37-4e45-be48-1b66617e259e" containerID="ad861a51acb0baa874dbd7238409301d609638ac8a31c29959dc0dfad3c4b8bc" exitCode=0 Jul 30 03:22:54.982222 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.982074 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" Jul 30 03:22:54.982222 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.982115 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" event={"ID":"555c2dc9-cd37-4e45-be48-1b66617e259e","Type":"ContainerDied","Data":"ad861a51acb0baa874dbd7238409301d609638ac8a31c29959dc0dfad3c4b8bc"} Jul 30 03:22:54.982222 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.982168 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x" event={"ID":"555c2dc9-cd37-4e45-be48-1b66617e259e","Type":"ContainerDied","Data":"ebf0822340e7bf40837772af6ecdb4b30b91fc3a44da51b5c7a07bc8ec08d42a"} Jul 30 03:22:54.982222 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.982198 2566 scope.go:117] "RemoveContainer" containerID="ad861a51acb0baa874dbd7238409301d609638ac8a31c29959dc0dfad3c4b8bc" Jul 30 03:22:54.983554 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.983532 2566 generic.go:356] "Generic (PLEG): container finished" podID="423a59cf-a528-402c-97f9-4783e708ee9c" containerID="b2042fece18785b35b90728b813ec98f76735ae0c23867d7737c1ddfd90cd8c4" exitCode=0 Jul 30 03:22:54.983693 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.983589 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" Jul 30 03:22:54.983693 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.983635 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" event={"ID":"423a59cf-a528-402c-97f9-4783e708ee9c","Type":"ContainerDied","Data":"b2042fece18785b35b90728b813ec98f76735ae0c23867d7737c1ddfd90cd8c4"} Jul 30 03:22:54.983784 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.983707 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb" event={"ID":"423a59cf-a528-402c-97f9-4783e708ee9c","Type":"ContainerDied","Data":"91df3cd7396bce8f528136f2fa386edfdedca68eb35d5f543be4cd3ce0a7eaf7"} Jul 30 03:22:54.991420 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.991401 2566 scope.go:117] "RemoveContainer" containerID="ad861a51acb0baa874dbd7238409301d609638ac8a31c29959dc0dfad3c4b8bc" Jul 30 03:22:54.991695 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:22:54.991676 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad861a51acb0baa874dbd7238409301d609638ac8a31c29959dc0dfad3c4b8bc\": container with ID starting with ad861a51acb0baa874dbd7238409301d609638ac8a31c29959dc0dfad3c4b8bc not found: ID does not exist" containerID="ad861a51acb0baa874dbd7238409301d609638ac8a31c29959dc0dfad3c4b8bc" Jul 30 03:22:54.991790 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.991703 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad861a51acb0baa874dbd7238409301d609638ac8a31c29959dc0dfad3c4b8bc"} err="failed to get container status \"ad861a51acb0baa874dbd7238409301d609638ac8a31c29959dc0dfad3c4b8bc\": rpc error: code = NotFound desc = could not find container \"ad861a51acb0baa874dbd7238409301d609638ac8a31c29959dc0dfad3c4b8bc\": container with ID starting with ad861a51acb0baa874dbd7238409301d609638ac8a31c29959dc0dfad3c4b8bc not found: ID does not exist" Jul 30 03:22:54.991790 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.991720 2566 scope.go:117] "RemoveContainer" containerID="b2042fece18785b35b90728b813ec98f76735ae0c23867d7737c1ddfd90cd8c4" Jul 30 03:22:54.999601 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.999580 2566 scope.go:117] "RemoveContainer" containerID="b2042fece18785b35b90728b813ec98f76735ae0c23867d7737c1ddfd90cd8c4" Jul 30 03:22:54.999888 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:22:54.999870 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2042fece18785b35b90728b813ec98f76735ae0c23867d7737c1ddfd90cd8c4\": container with ID starting with b2042fece18785b35b90728b813ec98f76735ae0c23867d7737c1ddfd90cd8c4 not found: ID does not exist" containerID="b2042fece18785b35b90728b813ec98f76735ae0c23867d7737c1ddfd90cd8c4" Jul 30 03:22:54.999963 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:54.999898 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2042fece18785b35b90728b813ec98f76735ae0c23867d7737c1ddfd90cd8c4"} err="failed to get container status \"b2042fece18785b35b90728b813ec98f76735ae0c23867d7737c1ddfd90cd8c4\": rpc error: code = NotFound desc = could not find container \"b2042fece18785b35b90728b813ec98f76735ae0c23867d7737c1ddfd90cd8c4\": container with ID starting with b2042fece18785b35b90728b813ec98f76735ae0c23867d7737c1ddfd90cd8c4 not found: ID does not exist" Jul 30 03:22:55.001752 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:55.001733 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x"] Jul 30 03:22:55.003749 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:55.003718 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-64f2f8-764c6d567b-vkp7x"] Jul 30 03:22:55.022941 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:55.022906 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb"] Jul 30 03:22:55.024841 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:55.024820 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-64f2f8-769ccd56cb-c8hlb"] Jul 30 03:22:56.853070 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:56.853037 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="423a59cf-a528-402c-97f9-4783e708ee9c" path="/var/lib/kubelet/pods/423a59cf-a528-402c-97f9-4783e708ee9c/volumes" Jul 30 03:22:56.853509 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:56.853374 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="555c2dc9-cd37-4e45-be48-1b66617e259e" path="/var/lib/kubelet/pods/555c2dc9-cd37-4e45-be48-1b66617e259e/volumes" Jul 30 03:22:58.729120 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:58.729088 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:22:59.101237 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:22:59.101204 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:23:04.892154 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:04.892111 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:23:04.951365 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:04.951333 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:23:05.156018 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:05.155938 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:23:05.418321 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:05.418244 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:23:24.154519 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.154487 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:23:24.184778 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.184748 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c"] Jul 30 03:23:24.184997 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.184944 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" podUID="b9e822c9-ba63-4f35-996a-b593edea73b1" containerName="payload-processing" containerID="cri-o://7d0597e72f2500f4e4d4f562d40c5d237a9b6e8a344e5aa73968ba2882c7b1c2" gracePeriod=30 Jul 30 03:23:24.190366 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.190337 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m"] Jul 30 03:23:24.190819 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.190768 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" podUID="8eae1883-5596-4335-aa03-d7e231ba8513" containerName="payload-pre-processing" containerID="cri-o://caf2550b8c031e31437cc40e3a92d5de452acd0c8935c67d8bccc2dd690ba901" gracePeriod=30 Jul 30 03:23:24.224047 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.224017 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:23:24.293638 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.293598 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:23:24.473498 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.473473 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" Jul 30 03:23:24.476499 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.476477 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" Jul 30 03:23:24.570381 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.570352 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eae1883-5596-4335-aa03-d7e231ba8513-plugins-config-volume\") pod \"8eae1883-5596-4335-aa03-d7e231ba8513\" (UID: \"8eae1883-5596-4335-aa03-d7e231ba8513\") " Jul 30 03:23:24.570529 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.570425 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvf29\" (UniqueName: \"kubernetes.io/projected/b9e822c9-ba63-4f35-996a-b593edea73b1-kube-api-access-cvf29\") pod \"b9e822c9-ba63-4f35-996a-b593edea73b1\" (UID: \"b9e822c9-ba63-4f35-996a-b593edea73b1\") " Jul 30 03:23:24.570609 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.570584 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9e822c9-ba63-4f35-996a-b593edea73b1-plugins-config-volume\") pod \"b9e822c9-ba63-4f35-996a-b593edea73b1\" (UID: \"b9e822c9-ba63-4f35-996a-b593edea73b1\") " Jul 30 03:23:24.570690 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.570623 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r88pq\" (UniqueName: \"kubernetes.io/projected/8eae1883-5596-4335-aa03-d7e231ba8513-kube-api-access-r88pq\") pod \"8eae1883-5596-4335-aa03-d7e231ba8513\" (UID: \"8eae1883-5596-4335-aa03-d7e231ba8513\") " Jul 30 03:23:24.570849 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.570802 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eae1883-5596-4335-aa03-d7e231ba8513-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8eae1883-5596-4335-aa03-d7e231ba8513" (UID: "8eae1883-5596-4335-aa03-d7e231ba8513"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:23:24.570983 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.570967 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9e822c9-ba63-4f35-996a-b593edea73b1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b9e822c9-ba63-4f35-996a-b593edea73b1" (UID: "b9e822c9-ba63-4f35-996a-b593edea73b1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:23:24.571056 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.571037 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eae1883-5596-4335-aa03-d7e231ba8513-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:23:24.571101 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.571065 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9e822c9-ba63-4f35-996a-b593edea73b1-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:23:24.572603 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.572576 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9e822c9-ba63-4f35-996a-b593edea73b1-kube-api-access-cvf29" (OuterVolumeSpecName: "kube-api-access-cvf29") pod "b9e822c9-ba63-4f35-996a-b593edea73b1" (UID: "b9e822c9-ba63-4f35-996a-b593edea73b1"). InnerVolumeSpecName "kube-api-access-cvf29". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:23:24.572715 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.572602 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eae1883-5596-4335-aa03-d7e231ba8513-kube-api-access-r88pq" (OuterVolumeSpecName: "kube-api-access-r88pq") pod "8eae1883-5596-4335-aa03-d7e231ba8513" (UID: "8eae1883-5596-4335-aa03-d7e231ba8513"). InnerVolumeSpecName "kube-api-access-r88pq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:23:24.672343 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.672318 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvf29\" (UniqueName: \"kubernetes.io/projected/b9e822c9-ba63-4f35-996a-b593edea73b1-kube-api-access-cvf29\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:23:24.672343 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:24.672339 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r88pq\" (UniqueName: \"kubernetes.io/projected/8eae1883-5596-4335-aa03-d7e231ba8513-kube-api-access-r88pq\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:23:25.137706 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.137581 2566 generic.go:356] "Generic (PLEG): container finished" podID="8eae1883-5596-4335-aa03-d7e231ba8513" containerID="caf2550b8c031e31437cc40e3a92d5de452acd0c8935c67d8bccc2dd690ba901" exitCode=0 Jul 30 03:23:25.137706 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.137638 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" Jul 30 03:23:25.137706 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.137693 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" event={"ID":"8eae1883-5596-4335-aa03-d7e231ba8513","Type":"ContainerDied","Data":"caf2550b8c031e31437cc40e3a92d5de452acd0c8935c67d8bccc2dd690ba901"} Jul 30 03:23:25.137958 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.137752 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m" event={"ID":"8eae1883-5596-4335-aa03-d7e231ba8513","Type":"ContainerDied","Data":"be94b44747ff587cf5e10db285ab121e11c37302349e4512499c9d8500ef3d7f"} Jul 30 03:23:25.137958 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.137785 2566 scope.go:117] "RemoveContainer" containerID="caf2550b8c031e31437cc40e3a92d5de452acd0c8935c67d8bccc2dd690ba901" Jul 30 03:23:25.139123 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.139098 2566 generic.go:356] "Generic (PLEG): container finished" podID="b9e822c9-ba63-4f35-996a-b593edea73b1" containerID="7d0597e72f2500f4e4d4f562d40c5d237a9b6e8a344e5aa73968ba2882c7b1c2" exitCode=0 Jul 30 03:23:25.139225 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.139130 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" event={"ID":"b9e822c9-ba63-4f35-996a-b593edea73b1","Type":"ContainerDied","Data":"7d0597e72f2500f4e4d4f562d40c5d237a9b6e8a344e5aa73968ba2882c7b1c2"} Jul 30 03:23:25.139225 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.139166 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" Jul 30 03:23:25.139225 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.139184 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c" event={"ID":"b9e822c9-ba63-4f35-996a-b593edea73b1","Type":"ContainerDied","Data":"128b716d15e4fe7e283ad07895d4836051f9b037342f7dd4dba94425d54ced3f"} Jul 30 03:23:25.146989 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.146968 2566 scope.go:117] "RemoveContainer" containerID="caf2550b8c031e31437cc40e3a92d5de452acd0c8935c67d8bccc2dd690ba901" Jul 30 03:23:25.147232 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:23:25.147213 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caf2550b8c031e31437cc40e3a92d5de452acd0c8935c67d8bccc2dd690ba901\": container with ID starting with caf2550b8c031e31437cc40e3a92d5de452acd0c8935c67d8bccc2dd690ba901 not found: ID does not exist" containerID="caf2550b8c031e31437cc40e3a92d5de452acd0c8935c67d8bccc2dd690ba901" Jul 30 03:23:25.147318 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.147237 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caf2550b8c031e31437cc40e3a92d5de452acd0c8935c67d8bccc2dd690ba901"} err="failed to get container status \"caf2550b8c031e31437cc40e3a92d5de452acd0c8935c67d8bccc2dd690ba901\": rpc error: code = NotFound desc = could not find container \"caf2550b8c031e31437cc40e3a92d5de452acd0c8935c67d8bccc2dd690ba901\": container with ID starting with caf2550b8c031e31437cc40e3a92d5de452acd0c8935c67d8bccc2dd690ba901 not found: ID does not exist" Jul 30 03:23:25.147318 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.147252 2566 scope.go:117] "RemoveContainer" containerID="7d0597e72f2500f4e4d4f562d40c5d237a9b6e8a344e5aa73968ba2882c7b1c2" Jul 30 03:23:25.154992 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.154974 2566 scope.go:117] "RemoveContainer" containerID="7d0597e72f2500f4e4d4f562d40c5d237a9b6e8a344e5aa73968ba2882c7b1c2" Jul 30 03:23:25.155280 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:23:25.155226 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d0597e72f2500f4e4d4f562d40c5d237a9b6e8a344e5aa73968ba2882c7b1c2\": container with ID starting with 7d0597e72f2500f4e4d4f562d40c5d237a9b6e8a344e5aa73968ba2882c7b1c2 not found: ID does not exist" containerID="7d0597e72f2500f4e4d4f562d40c5d237a9b6e8a344e5aa73968ba2882c7b1c2" Jul 30 03:23:25.155280 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.155246 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d0597e72f2500f4e4d4f562d40c5d237a9b6e8a344e5aa73968ba2882c7b1c2"} err="failed to get container status \"7d0597e72f2500f4e4d4f562d40c5d237a9b6e8a344e5aa73968ba2882c7b1c2\": rpc error: code = NotFound desc = could not find container \"7d0597e72f2500f4e4d4f562d40c5d237a9b6e8a344e5aa73968ba2882c7b1c2\": container with ID starting with 7d0597e72f2500f4e4d4f562d40c5d237a9b6e8a344e5aa73968ba2882c7b1c2 not found: ID does not exist" Jul 30 03:23:25.156694 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.156672 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c"] Jul 30 03:23:25.160953 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.160932 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2cad57-6c5f644974-lhr2c"] Jul 30 03:23:25.170231 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.170207 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m"] Jul 30 03:23:25.176198 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:25.176178 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2cad57-ffd86f556-cgc4m"] Jul 30 03:23:26.852982 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:26.852948 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eae1883-5596-4335-aa03-d7e231ba8513" path="/var/lib/kubelet/pods/8eae1883-5596-4335-aa03-d7e231ba8513/volumes" Jul 30 03:23:26.853350 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:26.853299 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9e822c9-ba63-4f35-996a-b593edea73b1" path="/var/lib/kubelet/pods/b9e822c9-ba63-4f35-996a-b593edea73b1/volumes" Jul 30 03:23:28.696024 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:28.695993 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:23:29.064249 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:29.064218 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:23:34.129049 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:23:34.129014 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:10.405554 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:10.405521 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:11.317028 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:11.316997 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:16.950823 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:16.950713 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:17.057928 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:17.057894 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:17.152354 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:17.152321 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:17.250945 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:17.250912 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:17.466084 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:17.466038 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:19.983813 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:19.983778 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:20.083858 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:20.083808 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:28.021160 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.021125 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv"] Jul 30 03:24:28.021627 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.021394 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" podUID="accfe378-7122-4d32-9b87-ff9718371bce" containerName="payload-processing" containerID="cri-o://14905c0cf0dbef8aa580883c0cdce081e78f0eff68cc0d4dd6d00a004a7e692b" gracePeriod=30 Jul 30 03:24:28.031508 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.031482 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5"] Jul 30 03:24:28.031906 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.031858 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" podUID="c220c6bd-7c2c-4077-8783-30e1c1c60653" containerName="payload-pre-processing" containerID="cri-o://06aaf1459fcff5ae2c4fb7affa8cb3ef0a4a8ac2651a3a863a8c9e8a2e3ef40c" gracePeriod=30 Jul 30 03:24:28.033887 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.033827 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:28.127084 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.127052 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:28.312544 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.312516 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" Jul 30 03:24:28.315806 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.315782 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" Jul 30 03:24:28.412421 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.412384 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/accfe378-7122-4d32-9b87-ff9718371bce-plugins-config-volume\") pod \"accfe378-7122-4d32-9b87-ff9718371bce\" (UID: \"accfe378-7122-4d32-9b87-ff9718371bce\") " Jul 30 03:24:28.412576 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.412442 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58sml\" (UniqueName: \"kubernetes.io/projected/accfe378-7122-4d32-9b87-ff9718371bce-kube-api-access-58sml\") pod \"accfe378-7122-4d32-9b87-ff9718371bce\" (UID: \"accfe378-7122-4d32-9b87-ff9718371bce\") " Jul 30 03:24:28.412576 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.412486 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c220c6bd-7c2c-4077-8783-30e1c1c60653-plugins-config-volume\") pod \"c220c6bd-7c2c-4077-8783-30e1c1c60653\" (UID: \"c220c6bd-7c2c-4077-8783-30e1c1c60653\") " Jul 30 03:24:28.412576 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.412509 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xrvx\" (UniqueName: \"kubernetes.io/projected/c220c6bd-7c2c-4077-8783-30e1c1c60653-kube-api-access-4xrvx\") pod \"c220c6bd-7c2c-4077-8783-30e1c1c60653\" (UID: \"c220c6bd-7c2c-4077-8783-30e1c1c60653\") " Jul 30 03:24:28.412849 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.412824 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/accfe378-7122-4d32-9b87-ff9718371bce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "accfe378-7122-4d32-9b87-ff9718371bce" (UID: "accfe378-7122-4d32-9b87-ff9718371bce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:24:28.412896 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.412858 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c220c6bd-7c2c-4077-8783-30e1c1c60653-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c220c6bd-7c2c-4077-8783-30e1c1c60653" (UID: "c220c6bd-7c2c-4077-8783-30e1c1c60653"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:24:28.414583 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.414552 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c220c6bd-7c2c-4077-8783-30e1c1c60653-kube-api-access-4xrvx" (OuterVolumeSpecName: "kube-api-access-4xrvx") pod "c220c6bd-7c2c-4077-8783-30e1c1c60653" (UID: "c220c6bd-7c2c-4077-8783-30e1c1c60653"). InnerVolumeSpecName "kube-api-access-4xrvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:24:28.414583 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.414564 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/accfe378-7122-4d32-9b87-ff9718371bce-kube-api-access-58sml" (OuterVolumeSpecName: "kube-api-access-58sml") pod "accfe378-7122-4d32-9b87-ff9718371bce" (UID: "accfe378-7122-4d32-9b87-ff9718371bce"). InnerVolumeSpecName "kube-api-access-58sml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:24:28.446834 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.446807 2566 generic.go:356] "Generic (PLEG): container finished" podID="c220c6bd-7c2c-4077-8783-30e1c1c60653" containerID="06aaf1459fcff5ae2c4fb7affa8cb3ef0a4a8ac2651a3a863a8c9e8a2e3ef40c" exitCode=0 Jul 30 03:24:28.446942 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.446850 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" Jul 30 03:24:28.446942 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.446893 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" event={"ID":"c220c6bd-7c2c-4077-8783-30e1c1c60653","Type":"ContainerDied","Data":"06aaf1459fcff5ae2c4fb7affa8cb3ef0a4a8ac2651a3a863a8c9e8a2e3ef40c"} Jul 30 03:24:28.447050 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.446946 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5" event={"ID":"c220c6bd-7c2c-4077-8783-30e1c1c60653","Type":"ContainerDied","Data":"3c7cb1f1629005d37da1ecafb2c2f3099f4fb860c0fa6dcfaf4bfd1667bc7e5e"} Jul 30 03:24:28.447050 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.446969 2566 scope.go:117] "RemoveContainer" containerID="06aaf1459fcff5ae2c4fb7affa8cb3ef0a4a8ac2651a3a863a8c9e8a2e3ef40c" Jul 30 03:24:28.448261 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.448238 2566 generic.go:356] "Generic (PLEG): container finished" podID="accfe378-7122-4d32-9b87-ff9718371bce" containerID="14905c0cf0dbef8aa580883c0cdce081e78f0eff68cc0d4dd6d00a004a7e692b" exitCode=0 Jul 30 03:24:28.448361 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.448282 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" Jul 30 03:24:28.448361 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.448342 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" event={"ID":"accfe378-7122-4d32-9b87-ff9718371bce","Type":"ContainerDied","Data":"14905c0cf0dbef8aa580883c0cdce081e78f0eff68cc0d4dd6d00a004a7e692b"} Jul 30 03:24:28.448466 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.448400 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv" event={"ID":"accfe378-7122-4d32-9b87-ff9718371bce","Type":"ContainerDied","Data":"53975abbcbc505b206e39048dc7a80e416dc4b024d87b9167ad105093a611060"} Jul 30 03:24:28.456048 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.456032 2566 scope.go:117] "RemoveContainer" containerID="06aaf1459fcff5ae2c4fb7affa8cb3ef0a4a8ac2651a3a863a8c9e8a2e3ef40c" Jul 30 03:24:28.456432 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:24:28.456331 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06aaf1459fcff5ae2c4fb7affa8cb3ef0a4a8ac2651a3a863a8c9e8a2e3ef40c\": container with ID starting with 06aaf1459fcff5ae2c4fb7affa8cb3ef0a4a8ac2651a3a863a8c9e8a2e3ef40c not found: ID does not exist" containerID="06aaf1459fcff5ae2c4fb7affa8cb3ef0a4a8ac2651a3a863a8c9e8a2e3ef40c" Jul 30 03:24:28.456432 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.456365 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06aaf1459fcff5ae2c4fb7affa8cb3ef0a4a8ac2651a3a863a8c9e8a2e3ef40c"} err="failed to get container status \"06aaf1459fcff5ae2c4fb7affa8cb3ef0a4a8ac2651a3a863a8c9e8a2e3ef40c\": rpc error: code = NotFound desc = could not find container \"06aaf1459fcff5ae2c4fb7affa8cb3ef0a4a8ac2651a3a863a8c9e8a2e3ef40c\": container with ID starting with 06aaf1459fcff5ae2c4fb7affa8cb3ef0a4a8ac2651a3a863a8c9e8a2e3ef40c not found: ID does not exist" Jul 30 03:24:28.456432 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.456388 2566 scope.go:117] "RemoveContainer" containerID="14905c0cf0dbef8aa580883c0cdce081e78f0eff68cc0d4dd6d00a004a7e692b" Jul 30 03:24:28.464038 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.464021 2566 scope.go:117] "RemoveContainer" containerID="14905c0cf0dbef8aa580883c0cdce081e78f0eff68cc0d4dd6d00a004a7e692b" Jul 30 03:24:28.464281 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:24:28.464262 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14905c0cf0dbef8aa580883c0cdce081e78f0eff68cc0d4dd6d00a004a7e692b\": container with ID starting with 14905c0cf0dbef8aa580883c0cdce081e78f0eff68cc0d4dd6d00a004a7e692b not found: ID does not exist" containerID="14905c0cf0dbef8aa580883c0cdce081e78f0eff68cc0d4dd6d00a004a7e692b" Jul 30 03:24:28.464343 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.464286 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14905c0cf0dbef8aa580883c0cdce081e78f0eff68cc0d4dd6d00a004a7e692b"} err="failed to get container status \"14905c0cf0dbef8aa580883c0cdce081e78f0eff68cc0d4dd6d00a004a7e692b\": rpc error: code = NotFound desc = could not find container \"14905c0cf0dbef8aa580883c0cdce081e78f0eff68cc0d4dd6d00a004a7e692b\": container with ID starting with 14905c0cf0dbef8aa580883c0cdce081e78f0eff68cc0d4dd6d00a004a7e692b not found: ID does not exist" Jul 30 03:24:28.493683 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.493610 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv"] Jul 30 03:24:28.496554 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.496524 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f58ab6-7fc89b57c6-gfdwv"] Jul 30 03:24:28.502880 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.502860 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5"] Jul 30 03:24:28.507310 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.507289 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f58ab6-5bdb994d64-sj4c5"] Jul 30 03:24:28.513050 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.513034 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58sml\" (UniqueName: \"kubernetes.io/projected/accfe378-7122-4d32-9b87-ff9718371bce-kube-api-access-58sml\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:24:28.513112 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.513053 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c220c6bd-7c2c-4077-8783-30e1c1c60653-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:24:28.513112 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.513063 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xrvx\" (UniqueName: \"kubernetes.io/projected/c220c6bd-7c2c-4077-8783-30e1c1c60653-kube-api-access-4xrvx\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:24:28.513112 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.513073 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/accfe378-7122-4d32-9b87-ff9718371bce-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:24:28.852379 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.852347 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="accfe378-7122-4d32-9b87-ff9718371bce" path="/var/lib/kubelet/pods/accfe378-7122-4d32-9b87-ff9718371bce/volumes" Jul 30 03:24:28.852807 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:28.852787 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c220c6bd-7c2c-4077-8783-30e1c1c60653" path="/var/lib/kubelet/pods/c220c6bd-7c2c-4077-8783-30e1c1c60653/volumes" Jul 30 03:24:32.655828 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:32.655794 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:33.036630 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:33.036601 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:38.846706 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:38.846644 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:38.947600 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:38.947569 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:39.115558 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:39.115475 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:39.339415 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:39.339384 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:50.052460 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.052408 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:50.054523 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.054490 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-da5459-29756355-dnpmw"] Jul 30 03:24:50.056238 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.056215 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-da5459-29756355-dnpmw"] Jul 30 03:24:50.088990 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.088957 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx"] Jul 30 03:24:50.089239 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.089191 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" podUID="7f870d8d-754f-45fa-a558-504c58e33f1b" containerName="payload-processing" containerID="cri-o://6acba1e72f72f7889dcdbf0fd2a9ee181362896a436c7f5996c6bcc18762642f" gracePeriod=30 Jul 30 03:24:50.096450 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.096344 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8"] Jul 30 03:24:50.096591 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.096542 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" podUID="02be289f-3540-42f6-b73a-b0f7de0c9ec5" containerName="payload-pre-processing" containerID="cri-o://ca43c21e642ea1f8e0e97a285c0eb79ed62ffa68299c0dad5305b1fef76c2815" gracePeriod=30 Jul 30 03:24:50.134452 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.134428 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:50.207061 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.207037 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:50.376403 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.376377 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" Jul 30 03:24:50.381618 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.381596 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" Jul 30 03:24:50.498480 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.498448 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jg96c\" (UniqueName: \"kubernetes.io/projected/02be289f-3540-42f6-b73a-b0f7de0c9ec5-kube-api-access-jg96c\") pod \"02be289f-3540-42f6-b73a-b0f7de0c9ec5\" (UID: \"02be289f-3540-42f6-b73a-b0f7de0c9ec5\") " Jul 30 03:24:50.498645 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.498539 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f870d8d-754f-45fa-a558-504c58e33f1b-plugins-config-volume\") pod \"7f870d8d-754f-45fa-a558-504c58e33f1b\" (UID: \"7f870d8d-754f-45fa-a558-504c58e33f1b\") " Jul 30 03:24:50.498645 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.498572 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02be289f-3540-42f6-b73a-b0f7de0c9ec5-plugins-config-volume\") pod \"02be289f-3540-42f6-b73a-b0f7de0c9ec5\" (UID: \"02be289f-3540-42f6-b73a-b0f7de0c9ec5\") " Jul 30 03:24:50.498645 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.498630 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpq2q\" (UniqueName: \"kubernetes.io/projected/7f870d8d-754f-45fa-a558-504c58e33f1b-kube-api-access-wpq2q\") pod \"7f870d8d-754f-45fa-a558-504c58e33f1b\" (UID: \"7f870d8d-754f-45fa-a558-504c58e33f1b\") " Jul 30 03:24:50.498942 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.498919 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f870d8d-754f-45fa-a558-504c58e33f1b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7f870d8d-754f-45fa-a558-504c58e33f1b" (UID: "7f870d8d-754f-45fa-a558-504c58e33f1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:24:50.499044 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.499018 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02be289f-3540-42f6-b73a-b0f7de0c9ec5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "02be289f-3540-42f6-b73a-b0f7de0c9ec5" (UID: "02be289f-3540-42f6-b73a-b0f7de0c9ec5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:24:50.500603 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.500573 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02be289f-3540-42f6-b73a-b0f7de0c9ec5-kube-api-access-jg96c" (OuterVolumeSpecName: "kube-api-access-jg96c") pod "02be289f-3540-42f6-b73a-b0f7de0c9ec5" (UID: "02be289f-3540-42f6-b73a-b0f7de0c9ec5"). InnerVolumeSpecName "kube-api-access-jg96c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:24:50.500738 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.500677 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f870d8d-754f-45fa-a558-504c58e33f1b-kube-api-access-wpq2q" (OuterVolumeSpecName: "kube-api-access-wpq2q") pod "7f870d8d-754f-45fa-a558-504c58e33f1b" (UID: "7f870d8d-754f-45fa-a558-504c58e33f1b"). InnerVolumeSpecName "kube-api-access-wpq2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:24:50.556733 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.556705 2566 generic.go:356] "Generic (PLEG): container finished" podID="02be289f-3540-42f6-b73a-b0f7de0c9ec5" containerID="ca43c21e642ea1f8e0e97a285c0eb79ed62ffa68299c0dad5305b1fef76c2815" exitCode=0 Jul 30 03:24:50.556867 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.556751 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" Jul 30 03:24:50.556867 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.556793 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" event={"ID":"02be289f-3540-42f6-b73a-b0f7de0c9ec5","Type":"ContainerDied","Data":"ca43c21e642ea1f8e0e97a285c0eb79ed62ffa68299c0dad5305b1fef76c2815"} Jul 30 03:24:50.556867 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.556858 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8" event={"ID":"02be289f-3540-42f6-b73a-b0f7de0c9ec5","Type":"ContainerDied","Data":"5010d45a9e777c16ba279c713f118ad103d161fef5ccabf846f2dd85d08c7e2e"} Jul 30 03:24:50.557041 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.556884 2566 scope.go:117] "RemoveContainer" containerID="ca43c21e642ea1f8e0e97a285c0eb79ed62ffa68299c0dad5305b1fef76c2815" Jul 30 03:24:50.558306 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.558275 2566 generic.go:356] "Generic (PLEG): container finished" podID="7f870d8d-754f-45fa-a558-504c58e33f1b" containerID="6acba1e72f72f7889dcdbf0fd2a9ee181362896a436c7f5996c6bcc18762642f" exitCode=0 Jul 30 03:24:50.558426 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.558310 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" event={"ID":"7f870d8d-754f-45fa-a558-504c58e33f1b","Type":"ContainerDied","Data":"6acba1e72f72f7889dcdbf0fd2a9ee181362896a436c7f5996c6bcc18762642f"} Jul 30 03:24:50.558426 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.558297 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" Jul 30 03:24:50.558426 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.558354 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx" event={"ID":"7f870d8d-754f-45fa-a558-504c58e33f1b","Type":"ContainerDied","Data":"5621b494db9fc17e250ec15f7981b27b3ab7c6acc559de65bc3d4bd4f0d041a8"} Jul 30 03:24:50.565852 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.565828 2566 scope.go:117] "RemoveContainer" containerID="ca43c21e642ea1f8e0e97a285c0eb79ed62ffa68299c0dad5305b1fef76c2815" Jul 30 03:24:50.566216 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:24:50.566191 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca43c21e642ea1f8e0e97a285c0eb79ed62ffa68299c0dad5305b1fef76c2815\": container with ID starting with ca43c21e642ea1f8e0e97a285c0eb79ed62ffa68299c0dad5305b1fef76c2815 not found: ID does not exist" containerID="ca43c21e642ea1f8e0e97a285c0eb79ed62ffa68299c0dad5305b1fef76c2815" Jul 30 03:24:50.566317 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.566225 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca43c21e642ea1f8e0e97a285c0eb79ed62ffa68299c0dad5305b1fef76c2815"} err="failed to get container status \"ca43c21e642ea1f8e0e97a285c0eb79ed62ffa68299c0dad5305b1fef76c2815\": rpc error: code = NotFound desc = could not find container \"ca43c21e642ea1f8e0e97a285c0eb79ed62ffa68299c0dad5305b1fef76c2815\": container with ID starting with ca43c21e642ea1f8e0e97a285c0eb79ed62ffa68299c0dad5305b1fef76c2815 not found: ID does not exist" Jul 30 03:24:50.566317 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.566248 2566 scope.go:117] "RemoveContainer" containerID="6acba1e72f72f7889dcdbf0fd2a9ee181362896a436c7f5996c6bcc18762642f" Jul 30 03:24:50.574883 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.574866 2566 scope.go:117] "RemoveContainer" containerID="6acba1e72f72f7889dcdbf0fd2a9ee181362896a436c7f5996c6bcc18762642f" Jul 30 03:24:50.575116 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:24:50.575100 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6acba1e72f72f7889dcdbf0fd2a9ee181362896a436c7f5996c6bcc18762642f\": container with ID starting with 6acba1e72f72f7889dcdbf0fd2a9ee181362896a436c7f5996c6bcc18762642f not found: ID does not exist" containerID="6acba1e72f72f7889dcdbf0fd2a9ee181362896a436c7f5996c6bcc18762642f" Jul 30 03:24:50.575178 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.575120 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6acba1e72f72f7889dcdbf0fd2a9ee181362896a436c7f5996c6bcc18762642f"} err="failed to get container status \"6acba1e72f72f7889dcdbf0fd2a9ee181362896a436c7f5996c6bcc18762642f\": rpc error: code = NotFound desc = could not find container \"6acba1e72f72f7889dcdbf0fd2a9ee181362896a436c7f5996c6bcc18762642f\": container with ID starting with 6acba1e72f72f7889dcdbf0fd2a9ee181362896a436c7f5996c6bcc18762642f not found: ID does not exist" Jul 30 03:24:50.580030 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.580007 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8"] Jul 30 03:24:50.585829 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.585807 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-da5459-5467cf7ff6-gh7m8"] Jul 30 03:24:50.595537 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.595518 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx"] Jul 30 03:24:50.599460 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.599414 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-da5459-5d75568f98-xzfrx"] Jul 30 03:24:50.599795 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.599776 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f870d8d-754f-45fa-a558-504c58e33f1b-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:24:50.599843 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.599800 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02be289f-3540-42f6-b73a-b0f7de0c9ec5-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:24:50.599843 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.599810 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpq2q\" (UniqueName: \"kubernetes.io/projected/7f870d8d-754f-45fa-a558-504c58e33f1b-kube-api-access-wpq2q\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:24:50.599843 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.599818 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg96c\" (UniqueName: \"kubernetes.io/projected/02be289f-3540-42f6-b73a-b0f7de0c9ec5-kube-api-access-jg96c\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:24:50.852597 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.852519 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02be289f-3540-42f6-b73a-b0f7de0c9ec5" path="/var/lib/kubelet/pods/02be289f-3540-42f6-b73a-b0f7de0c9ec5/volumes" Jul 30 03:24:50.852900 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.852878 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d111f3c-380b-483b-a585-a2b97b48eaf2" path="/var/lib/kubelet/pods/5d111f3c-380b-483b-a585-a2b97b48eaf2/volumes" Jul 30 03:24:50.853219 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:50.853201 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f870d8d-754f-45fa-a558-504c58e33f1b" path="/var/lib/kubelet/pods/7f870d8d-754f-45fa-a558-504c58e33f1b/volumes" Jul 30 03:24:53.199921 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:53.199895 2566 scope.go:117] "RemoveContainer" containerID="615ead914a4191732ad6012368c7b940efaa15863cb74406a53f95e6845708ce" Jul 30 03:24:54.562266 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:54.562231 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:24:54.926781 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:24:54.926705 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:00.777403 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:00.777366 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:00.878006 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:00.877969 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:01.051948 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.051861 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:01.268073 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.268041 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:01.373148 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373045 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj"] Jul 30 03:25:01.373508 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373492 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02be289f-3540-42f6-b73a-b0f7de0c9ec5" containerName="payload-pre-processing" Jul 30 03:25:01.373561 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373510 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="02be289f-3540-42f6-b73a-b0f7de0c9ec5" containerName="payload-pre-processing" Jul 30 03:25:01.373561 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373523 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="accfe378-7122-4d32-9b87-ff9718371bce" containerName="payload-processing" Jul 30 03:25:01.373561 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373528 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="accfe378-7122-4d32-9b87-ff9718371bce" containerName="payload-processing" Jul 30 03:25:01.373561 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373537 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f870d8d-754f-45fa-a558-504c58e33f1b" containerName="payload-processing" Jul 30 03:25:01.373561 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373543 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f870d8d-754f-45fa-a558-504c58e33f1b" containerName="payload-processing" Jul 30 03:25:01.373561 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373550 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c220c6bd-7c2c-4077-8783-30e1c1c60653" containerName="payload-pre-processing" Jul 30 03:25:01.373561 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373556 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c220c6bd-7c2c-4077-8783-30e1c1c60653" containerName="payload-pre-processing" Jul 30 03:25:01.373561 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373562 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="423a59cf-a528-402c-97f9-4783e708ee9c" containerName="payload-pre-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373568 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="423a59cf-a528-402c-97f9-4783e708ee9c" containerName="payload-pre-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373573 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="555c2dc9-cd37-4e45-be48-1b66617e259e" containerName="payload-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373579 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="555c2dc9-cd37-4e45-be48-1b66617e259e" containerName="payload-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373589 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eae1883-5596-4335-aa03-d7e231ba8513" containerName="payload-pre-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373594 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eae1883-5596-4335-aa03-d7e231ba8513" containerName="payload-pre-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373600 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9e822c9-ba63-4f35-996a-b593edea73b1" containerName="payload-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373606 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9e822c9-ba63-4f35-996a-b593edea73b1" containerName="payload-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373676 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f870d8d-754f-45fa-a558-504c58e33f1b" containerName="payload-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373684 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="555c2dc9-cd37-4e45-be48-1b66617e259e" containerName="payload-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373691 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="423a59cf-a528-402c-97f9-4783e708ee9c" containerName="payload-pre-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373698 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="accfe378-7122-4d32-9b87-ff9718371bce" containerName="payload-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373705 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9e822c9-ba63-4f35-996a-b593edea73b1" containerName="payload-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373711 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="02be289f-3540-42f6-b73a-b0f7de0c9ec5" containerName="payload-pre-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373717 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eae1883-5596-4335-aa03-d7e231ba8513" containerName="payload-pre-processing" Jul 30 03:25:01.373853 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.373723 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c220c6bd-7c2c-4077-8783-30e1c1c60653" containerName="payload-pre-processing" Jul 30 03:25:01.378764 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.378744 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj" Jul 30 03:25:01.381032 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.381010 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 03:25:01.381134 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.381085 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2npk7\"" Jul 30 03:25:01.384325 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.384302 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj"] Jul 30 03:25:01.391439 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.391413 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b5e9192-3735-463d-b569-3ecc34c3b69a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj\" (UID: \"8b5e9192-3735-463d-b569-3ecc34c3b69a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj" Jul 30 03:25:01.492490 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.492458 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b5e9192-3735-463d-b569-3ecc34c3b69a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj\" (UID: \"8b5e9192-3735-463d-b569-3ecc34c3b69a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj" Jul 30 03:25:01.493059 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.493042 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b5e9192-3735-463d-b569-3ecc34c3b69a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj\" (UID: \"8b5e9192-3735-463d-b569-3ecc34c3b69a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj" Jul 30 03:25:01.691387 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.691298 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj" Jul 30 03:25:01.818057 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:01.818029 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj"] Jul 30 03:25:01.821299 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:25:01.821266 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b5e9192_3735_463d_b569_3ecc34c3b69a.slice/crio-e82edf95ba8da9ec1a74d4c0f32161548dd846f7a133a28d2a347c7c70648688 WatchSource:0}: Error finding container e82edf95ba8da9ec1a74d4c0f32161548dd846f7a133a28d2a347c7c70648688: Status 404 returned error can't find the container with id e82edf95ba8da9ec1a74d4c0f32161548dd846f7a133a28d2a347c7c70648688 Jul 30 03:25:02.618509 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:02.618473 2566 generic.go:356] "Generic (PLEG): container finished" podID="8b5e9192-3735-463d-b569-3ecc34c3b69a" containerID="aa9b996c322bfb8a36b664c318863128c4f9c86c3055fc875c638ee8de513945" exitCode=0 Jul 30 03:25:02.618718 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:02.618559 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj" event={"ID":"8b5e9192-3735-463d-b569-3ecc34c3b69a","Type":"ContainerDied","Data":"aa9b996c322bfb8a36b664c318863128c4f9c86c3055fc875c638ee8de513945"} Jul 30 03:25:02.618718 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:02.618611 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj" event={"ID":"8b5e9192-3735-463d-b569-3ecc34c3b69a","Type":"ContainerStarted","Data":"e82edf95ba8da9ec1a74d4c0f32161548dd846f7a133a28d2a347c7c70648688"} Jul 30 03:25:03.760208 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:03.760181 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj" Jul 30 03:25:03.813683 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:03.813637 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b5e9192-3735-463d-b569-3ecc34c3b69a-maas-api-service-ca\") pod \"8b5e9192-3735-463d-b569-3ecc34c3b69a\" (UID: \"8b5e9192-3735-463d-b569-3ecc34c3b69a\") " Jul 30 03:25:03.813957 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:03.813935 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b5e9192-3735-463d-b569-3ecc34c3b69a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8b5e9192-3735-463d-b569-3ecc34c3b69a" (UID: "8b5e9192-3735-463d-b569-3ecc34c3b69a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:25:03.914896 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:03.914836 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b5e9192-3735-463d-b569-3ecc34c3b69a-maas-api-service-ca\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:25:04.629716 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:04.629680 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj" Jul 30 03:25:04.629716 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:04.629692 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj" event={"ID":"8b5e9192-3735-463d-b569-3ecc34c3b69a","Type":"ContainerDied","Data":"e82edf95ba8da9ec1a74d4c0f32161548dd846f7a133a28d2a347c7c70648688"} Jul 30 03:25:04.629985 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:04.629744 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e82edf95ba8da9ec1a74d4c0f32161548dd846f7a133a28d2a347c7c70648688" Jul 30 03:25:11.796267 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:11.796201 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:11.819133 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:11.819100 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc"] Jul 30 03:25:11.819398 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:11.819372 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" podUID="9db2523f-0d42-42fb-9eef-f5110e708ac3" containerName="payload-processing" containerID="cri-o://8f2b09551cf8025f830aa906757286ea00d2487504af6d63730ffa087a2d131d" gracePeriod=30 Jul 30 03:25:11.824760 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:11.824721 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp"] Jul 30 03:25:11.824998 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:11.824940 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" podUID="351b6c0c-d9e5-4176-a1af-06b5c67e7347" containerName="payload-pre-processing" containerID="cri-o://3757737e5e6d809041182aa849bba88a9cf5d4c71a0a8ead8c9cfa363a59ae98" gracePeriod=30 Jul 30 03:25:12.082263 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.082240 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" Jul 30 03:25:12.105216 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.105195 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" Jul 30 03:25:12.189607 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.189584 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jh8mr\" (UniqueName: \"kubernetes.io/projected/351b6c0c-d9e5-4176-a1af-06b5c67e7347-kube-api-access-jh8mr\") pod \"351b6c0c-d9e5-4176-a1af-06b5c67e7347\" (UID: \"351b6c0c-d9e5-4176-a1af-06b5c67e7347\") " Jul 30 03:25:12.189769 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.189616 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6n9g5\" (UniqueName: \"kubernetes.io/projected/9db2523f-0d42-42fb-9eef-f5110e708ac3-kube-api-access-6n9g5\") pod \"9db2523f-0d42-42fb-9eef-f5110e708ac3\" (UID: \"9db2523f-0d42-42fb-9eef-f5110e708ac3\") " Jul 30 03:25:12.189769 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.189698 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9db2523f-0d42-42fb-9eef-f5110e708ac3-plugins-config-volume\") pod \"9db2523f-0d42-42fb-9eef-f5110e708ac3\" (UID: \"9db2523f-0d42-42fb-9eef-f5110e708ac3\") " Jul 30 03:25:12.189896 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.189799 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351b6c0c-d9e5-4176-a1af-06b5c67e7347-plugins-config-volume\") pod \"351b6c0c-d9e5-4176-a1af-06b5c67e7347\" (UID: \"351b6c0c-d9e5-4176-a1af-06b5c67e7347\") " Jul 30 03:25:12.190140 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.190113 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9db2523f-0d42-42fb-9eef-f5110e708ac3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9db2523f-0d42-42fb-9eef-f5110e708ac3" (UID: "9db2523f-0d42-42fb-9eef-f5110e708ac3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:25:12.190212 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.190155 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/351b6c0c-d9e5-4176-a1af-06b5c67e7347-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "351b6c0c-d9e5-4176-a1af-06b5c67e7347" (UID: "351b6c0c-d9e5-4176-a1af-06b5c67e7347"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 03:25:12.191640 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.191618 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9db2523f-0d42-42fb-9eef-f5110e708ac3-kube-api-access-6n9g5" (OuterVolumeSpecName: "kube-api-access-6n9g5") pod "9db2523f-0d42-42fb-9eef-f5110e708ac3" (UID: "9db2523f-0d42-42fb-9eef-f5110e708ac3"). InnerVolumeSpecName "kube-api-access-6n9g5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:25:12.191689 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.191625 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/351b6c0c-d9e5-4176-a1af-06b5c67e7347-kube-api-access-jh8mr" (OuterVolumeSpecName: "kube-api-access-jh8mr") pod "351b6c0c-d9e5-4176-a1af-06b5c67e7347" (UID: "351b6c0c-d9e5-4176-a1af-06b5c67e7347"). InnerVolumeSpecName "kube-api-access-jh8mr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 03:25:12.291354 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.291329 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jh8mr\" (UniqueName: \"kubernetes.io/projected/351b6c0c-d9e5-4176-a1af-06b5c67e7347-kube-api-access-jh8mr\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:25:12.291354 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.291352 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n9g5\" (UniqueName: \"kubernetes.io/projected/9db2523f-0d42-42fb-9eef-f5110e708ac3-kube-api-access-6n9g5\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:25:12.291494 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.291363 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9db2523f-0d42-42fb-9eef-f5110e708ac3-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:25:12.291494 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.291373 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351b6c0c-d9e5-4176-a1af-06b5c67e7347-plugins-config-volume\") on node \"ip-10-0-131-5.ec2.internal\" DevicePath \"\"" Jul 30 03:25:12.672776 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.672731 2566 generic.go:356] "Generic (PLEG): container finished" podID="9db2523f-0d42-42fb-9eef-f5110e708ac3" containerID="8f2b09551cf8025f830aa906757286ea00d2487504af6d63730ffa087a2d131d" exitCode=0 Jul 30 03:25:12.672964 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.672782 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" Jul 30 03:25:12.672964 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.672796 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" event={"ID":"9db2523f-0d42-42fb-9eef-f5110e708ac3","Type":"ContainerDied","Data":"8f2b09551cf8025f830aa906757286ea00d2487504af6d63730ffa087a2d131d"} Jul 30 03:25:12.672964 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.672850 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc" event={"ID":"9db2523f-0d42-42fb-9eef-f5110e708ac3","Type":"ContainerDied","Data":"d166e2832724b6ae9a83c3a88cf3a158f6ba99195422838e7f790e3845753c27"} Jul 30 03:25:12.672964 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.672882 2566 scope.go:117] "RemoveContainer" containerID="8f2b09551cf8025f830aa906757286ea00d2487504af6d63730ffa087a2d131d" Jul 30 03:25:12.674339 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.674316 2566 generic.go:356] "Generic (PLEG): container finished" podID="351b6c0c-d9e5-4176-a1af-06b5c67e7347" containerID="3757737e5e6d809041182aa849bba88a9cf5d4c71a0a8ead8c9cfa363a59ae98" exitCode=0 Jul 30 03:25:12.674447 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.674346 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" Jul 30 03:25:12.674447 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.674380 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" event={"ID":"351b6c0c-d9e5-4176-a1af-06b5c67e7347","Type":"ContainerDied","Data":"3757737e5e6d809041182aa849bba88a9cf5d4c71a0a8ead8c9cfa363a59ae98"} Jul 30 03:25:12.674447 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.674416 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp" event={"ID":"351b6c0c-d9e5-4176-a1af-06b5c67e7347","Type":"ContainerDied","Data":"8c2e1742b67ecfa2f42c1a9ad714ff0ce2149e6c4b441b172d2518de62ceea6e"} Jul 30 03:25:12.683079 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.683049 2566 scope.go:117] "RemoveContainer" containerID="8f2b09551cf8025f830aa906757286ea00d2487504af6d63730ffa087a2d131d" Jul 30 03:25:12.688288 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:25:12.687960 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f2b09551cf8025f830aa906757286ea00d2487504af6d63730ffa087a2d131d\": container with ID starting with 8f2b09551cf8025f830aa906757286ea00d2487504af6d63730ffa087a2d131d not found: ID does not exist" containerID="8f2b09551cf8025f830aa906757286ea00d2487504af6d63730ffa087a2d131d" Jul 30 03:25:12.688288 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.688001 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2b09551cf8025f830aa906757286ea00d2487504af6d63730ffa087a2d131d"} err="failed to get container status \"8f2b09551cf8025f830aa906757286ea00d2487504af6d63730ffa087a2d131d\": rpc error: code = NotFound desc = could not find container \"8f2b09551cf8025f830aa906757286ea00d2487504af6d63730ffa087a2d131d\": container with ID starting with 8f2b09551cf8025f830aa906757286ea00d2487504af6d63730ffa087a2d131d not found: ID does not exist" Jul 30 03:25:12.688288 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.688027 2566 scope.go:117] "RemoveContainer" containerID="3757737e5e6d809041182aa849bba88a9cf5d4c71a0a8ead8c9cfa363a59ae98" Jul 30 03:25:12.696472 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.696456 2566 scope.go:117] "RemoveContainer" containerID="3757737e5e6d809041182aa849bba88a9cf5d4c71a0a8ead8c9cfa363a59ae98" Jul 30 03:25:12.696732 ip-10-0-131-5 kubenswrapper[2566]: E0730 03:25:12.696709 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3757737e5e6d809041182aa849bba88a9cf5d4c71a0a8ead8c9cfa363a59ae98\": container with ID starting with 3757737e5e6d809041182aa849bba88a9cf5d4c71a0a8ead8c9cfa363a59ae98 not found: ID does not exist" containerID="3757737e5e6d809041182aa849bba88a9cf5d4c71a0a8ead8c9cfa363a59ae98" Jul 30 03:25:12.696818 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.696737 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3757737e5e6d809041182aa849bba88a9cf5d4c71a0a8ead8c9cfa363a59ae98"} err="failed to get container status \"3757737e5e6d809041182aa849bba88a9cf5d4c71a0a8ead8c9cfa363a59ae98\": rpc error: code = NotFound desc = could not find container \"3757737e5e6d809041182aa849bba88a9cf5d4c71a0a8ead8c9cfa363a59ae98\": container with ID starting with 3757737e5e6d809041182aa849bba88a9cf5d4c71a0a8ead8c9cfa363a59ae98 not found: ID does not exist" Jul 30 03:25:12.703744 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.703723 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc"] Jul 30 03:25:12.707690 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.707655 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f2a805-5c5b5fd8b-4bkcc"] Jul 30 03:25:12.717267 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.717244 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp"] Jul 30 03:25:12.720767 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.720749 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f2a805-6bc876ff5b-ww6jp"] Jul 30 03:25:12.852643 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.852615 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="351b6c0c-d9e5-4176-a1af-06b5c67e7347" path="/var/lib/kubelet/pods/351b6c0c-d9e5-4176-a1af-06b5c67e7347/volumes" Jul 30 03:25:12.853000 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.852986 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9db2523f-0d42-42fb-9eef-f5110e708ac3" path="/var/lib/kubelet/pods/9db2523f-0d42-42fb-9eef-f5110e708ac3/volumes" Jul 30 03:25:12.965634 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:12.965568 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:16.445629 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:16.445593 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:16.446472 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:16.446448 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj"] Jul 30 03:25:16.448066 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:16.448042 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f2a805-nb2cj"] Jul 30 03:25:16.825622 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:16.825585 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:16.852124 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:16.852088 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b5e9192-3735-463d-b569-3ecc34c3b69a" path="/var/lib/kubelet/pods/8b5e9192-3735-463d-b569-3ecc34c3b69a/volumes" Jul 30 03:25:22.660797 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:22.660766 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:22.761339 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:22.761312 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:22.937898 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:22.937818 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:23.162969 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:23.162935 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:29.349964 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:29.349934 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-684d7c5d99-8qwsx_b5749fdd-546a-418f-80f3-73bc7330b6d7/manager/0.log" Jul 30 03:25:29.356477 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:29.356456 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-684d7c5d99-8qwsx_b5749fdd-546a-418f-80f3-73bc7330b6d7/copy-manifests/0.log" Jul 30 03:25:30.245933 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:30.245901 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29756355-jthnt_32d69bd9-c3f0-4c6c-8d69-19de3c31dabe/cleanup/0.log" Jul 30 03:25:32.370158 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:32.370126 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4gw6f_cb82db02-b05d-4a8a-b760-5187383f81c3/manager/0.log" Jul 30 03:25:32.828712 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:32.828682 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kctzd_2c6df948-791a-438a-bbe5-fdcc8108abbd/discovery/0.log" Jul 30 03:25:33.279128 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:33.279102 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-z6xs4_d61232e1-244c-407e-9342-f6ed6bf47758/payload-pre-processing/0.log" Jul 30 03:25:33.835588 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:33.835562 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-2ckdn_4e936a43-e407-49d0-a435-07ec83c11999/main/0.log" Jul 30 03:25:34.165823 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:34.165756 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5sztv_0bc2a06b-8835-4faf-bd1d-986eae9b769d/main/0.log" Jul 30 03:25:34.276548 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:34.276524 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-qwbk8_0357639b-a9c5-4f67-a6d4-3fa3208236a7/main/0.log" Jul 30 03:25:40.016164 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:40.016125 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:40.117056 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:40.117021 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" limit=1024 Jul 30 03:25:42.828644 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:42.828614 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dp2xm_1aca484e-f11d-45ae-acbb-e227a511b00c/global-pull-secret-syncer/0.log" Jul 30 03:25:42.918006 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:42.917977 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9l7r4_9f09be67-5de5-4dc5-927e-9b1536d384e9/konnectivity-agent/0.log" Jul 30 03:25:43.009889 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:43.009858 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-5.ec2.internal_a52f7c3fa8fcd9dbf280379edf7b0272/haproxy/0.log" Jul 30 03:25:47.851407 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:47.851327 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4gw6f_cb82db02-b05d-4a8a-b760-5187383f81c3/manager/0.log" Jul 30 03:25:49.677884 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:49.677861 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-dsvcf_361c8b92-f91e-4d15-9167-383cb9ef1c90/cluster-monitoring-operator/0.log" Jul 30 03:25:49.781090 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:49.781067 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-76d86795b9-jm949_bcdf28eb-92fa-4329-8ef3-141f5edba294/metrics-server/0.log" Jul 30 03:25:50.070891 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.070862 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t7kcf_dfcf4fa5-3952-4914-a9db-013f7be543b2/node-exporter/0.log" Jul 30 03:25:50.092532 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.092513 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t7kcf_dfcf4fa5-3952-4914-a9db-013f7be543b2/kube-rbac-proxy/0.log" Jul 30 03:25:50.113551 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.113533 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t7kcf_dfcf4fa5-3952-4914-a9db-013f7be543b2/init-textfile/0.log" Jul 30 03:25:50.143923 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.143902 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-882jd_97bc4540-f598-47e6-9a59-da24d7e49ec7/kube-rbac-proxy-main/0.log" Jul 30 03:25:50.183384 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.183366 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-882jd_97bc4540-f598-47e6-9a59-da24d7e49ec7/kube-rbac-proxy-self/0.log" Jul 30 03:25:50.200479 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.200458 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-882jd_97bc4540-f598-47e6-9a59-da24d7e49ec7/openshift-state-metrics/0.log" Jul 30 03:25:50.237278 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.237256 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b808203-246f-4454-b334-f47f6908eedb/prometheus/0.log" Jul 30 03:25:50.261186 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.261165 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b808203-246f-4454-b334-f47f6908eedb/config-reloader/0.log" Jul 30 03:25:50.279647 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.279621 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b808203-246f-4454-b334-f47f6908eedb/thanos-sidecar/0.log" Jul 30 03:25:50.298291 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.298274 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b808203-246f-4454-b334-f47f6908eedb/kube-rbac-proxy-web/0.log" Jul 30 03:25:50.317836 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.317819 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b808203-246f-4454-b334-f47f6908eedb/kube-rbac-proxy/0.log" Jul 30 03:25:50.338364 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.338312 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b808203-246f-4454-b334-f47f6908eedb/kube-rbac-proxy-thanos/0.log" Jul 30 03:25:50.357260 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.357243 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b808203-246f-4454-b334-f47f6908eedb/init-config-reloader/0.log" Jul 30 03:25:50.390786 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.390761 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-7hf6p_ee72ae37-8b70-41f4-9fe5-96c3e4bb4291/prometheus-operator/0.log" Jul 30 03:25:50.412620 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:50.412591 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-7hf6p_ee72ae37-8b70-41f4-9fe5-96c3e4bb4291/kube-rbac-proxy/0.log" Jul 30 03:25:51.953893 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:51.953824 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-rr58w_1b389afb-4064-40ac-a473-89da51b1f974/networking-console-plugin/0.log" Jul 30 03:25:53.511062 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.511031 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87"] Jul 30 03:25:53.511438 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.511370 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b5e9192-3735-463d-b569-3ecc34c3b69a" containerName="revoke-keys" Jul 30 03:25:53.511438 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.511381 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b5e9192-3735-463d-b569-3ecc34c3b69a" containerName="revoke-keys" Jul 30 03:25:53.511438 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.511399 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db2523f-0d42-42fb-9eef-f5110e708ac3" containerName="payload-processing" Jul 30 03:25:53.511438 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.511405 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db2523f-0d42-42fb-9eef-f5110e708ac3" containerName="payload-processing" Jul 30 03:25:53.511438 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.511421 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="351b6c0c-d9e5-4176-a1af-06b5c67e7347" containerName="payload-pre-processing" Jul 30 03:25:53.511438 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.511426 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="351b6c0c-d9e5-4176-a1af-06b5c67e7347" containerName="payload-pre-processing" Jul 30 03:25:53.511624 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.511475 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9db2523f-0d42-42fb-9eef-f5110e708ac3" containerName="payload-processing" Jul 30 03:25:53.511624 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.511484 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b5e9192-3735-463d-b569-3ecc34c3b69a" containerName="revoke-keys" Jul 30 03:25:53.511624 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.511493 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="351b6c0c-d9e5-4176-a1af-06b5c67e7347" containerName="payload-pre-processing" Jul 30 03:25:53.515993 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.515977 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.518330 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.518310 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-trtdw\"/\"openshift-service-ca.crt\"" Jul 30 03:25:53.519658 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.519340 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-trtdw\"/\"default-dockercfg-6rs4c\"" Jul 30 03:25:53.519658 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.519468 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-trtdw\"/\"kube-root-ca.crt\"" Jul 30 03:25:53.521912 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.521889 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87"] Jul 30 03:25:53.532417 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.532396 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-sys\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.532504 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.532484 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-podres\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.532557 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.532541 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxkxc\" (UniqueName: \"kubernetes.io/projected/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-kube-api-access-zxkxc\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.532607 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.532593 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-proc\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.532703 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.532674 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-lib-modules\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.633840 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.633813 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-lib-modules\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.633936 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.633859 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-sys\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.633936 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.633911 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-podres\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.633936 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.633926 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-sys\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.634060 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.633959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxkxc\" (UniqueName: \"kubernetes.io/projected/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-kube-api-access-zxkxc\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.634060 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.633992 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-proc\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.634060 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.634015 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-lib-modules\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.634060 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.634042 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-podres\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.634199 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.634085 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-proc\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.642106 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.642081 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxkxc\" (UniqueName: \"kubernetes.io/projected/f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44-kube-api-access-zxkxc\") pod \"perf-node-gather-daemonset-6wx87\" (UID: \"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44\") " pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.822029 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.821965 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-j7lbt_e0b136e6-cc5e-4f07-9e0b-ec7cfd534e84/volume-data-source-validator/0.log" Jul 30 03:25:53.826469 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.826451 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:53.956264 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:53.956238 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87"] Jul 30 03:25:53.958475 ip-10-0-131-5 kubenswrapper[2566]: W0730 03:25:53.958443 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf0ec8362_8ad5_4d97_ab15_73f3fc3ebb44.slice/crio-d159784fa46d9d070d5c119aa43bff73003de1035d98a555ba08aa57a5060034 WatchSource:0}: Error finding container d159784fa46d9d070d5c119aa43bff73003de1035d98a555ba08aa57a5060034: Status 404 returned error can't find the container with id d159784fa46d9d070d5c119aa43bff73003de1035d98a555ba08aa57a5060034 Jul 30 03:25:54.876816 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:54.876782 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" event={"ID":"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44","Type":"ContainerStarted","Data":"80ef28e135fd1fa36cbd839690d4c80298ecdadfd6b1c7d0faa0afcf570f5952"} Jul 30 03:25:54.877191 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:54.876832 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" event={"ID":"f0ec8362-8ad5-4d97-ab15-73f3fc3ebb44","Type":"ContainerStarted","Data":"d159784fa46d9d070d5c119aa43bff73003de1035d98a555ba08aa57a5060034"} Jul 30 03:25:54.877191 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:54.876893 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:25:54.892171 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:54.892127 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" podStartSLOduration=1.892113884 podStartE2EDuration="1.892113884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 03:25:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 03:25:54.890874259 +0000 UTC m=+3602.569678254" watchObservedRunningTime="2026-07-30 03:25:54.892113884 +0000 UTC m=+3602.570917879" Jul 30 03:25:55.011234 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:55.011202 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7rjk5_d67b207e-c987-490a-9051-3006d810dee3/dns/0.log" Jul 30 03:25:55.029900 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:55.029879 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7rjk5_d67b207e-c987-490a-9051-3006d810dee3/kube-rbac-proxy/0.log" Jul 30 03:25:55.050162 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:55.050141 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-84dj8_2a06f56e-3de8-45af-b3aa-da354ff4e895/dns-node-resolver/0.log" Jul 30 03:25:55.689586 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:55.689556 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tsrrd_735cd8d6-2250-4b53-abef-118129c12350/node-ca/0.log" Jul 30 03:25:56.749552 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:56.749527 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kctzd_2c6df948-791a-438a-bbe5-fdcc8108abbd/discovery/0.log" Jul 30 03:25:56.879590 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:56.879561 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-z6xs4_d61232e1-244c-407e-9342-f6ed6bf47758/payload-pre-processing/0.log" Jul 30 03:25:57.761340 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:57.761311 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7pj54_b9d2ebcc-f9c2-414e-8bb7-66ca254e5fe0/serve-healthcheck-canary/0.log" Jul 30 03:25:58.245345 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:58.245275 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2rwzh_5452b49a-0c09-48bb-8cd3-1d4b19ce9473/kube-rbac-proxy/0.log" Jul 30 03:25:58.268950 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:58.268920 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2rwzh_5452b49a-0c09-48bb-8cd3-1d4b19ce9473/exporter/0.log" Jul 30 03:25:58.292176 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:25:58.292157 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2rwzh_5452b49a-0c09-48bb-8cd3-1d4b19ce9473/extractor/0.log" Jul 30 03:26:00.892269 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:00.892233 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-trtdw/perf-node-gather-daemonset-6wx87" Jul 30 03:26:01.069360 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:01.069333 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-684d7c5d99-8qwsx_b5749fdd-546a-418f-80f3-73bc7330b6d7/manager/0.log" Jul 30 03:26:01.108469 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:01.108442 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-684d7c5d99-8qwsx_b5749fdd-546a-418f-80f3-73bc7330b6d7/copy-manifests/0.log" Jul 30 03:26:02.589143 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:02.589119 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5bb854bdc9-2hb5h_b24dcdd4-f298-4e4e-924b-3cd2b0405e45/manager/0.log" Jul 30 03:26:04.037176 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:04.037145 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kctzd_2c6df948-791a-438a-bbe5-fdcc8108abbd/discovery/0.log" Jul 30 03:26:04.137826 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:04.137797 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-z6xs4_d61232e1-244c-407e-9342-f6ed6bf47758/payload-pre-processing/0.log" Jul 30 03:26:05.848870 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:05.848837 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kctzd_2c6df948-791a-438a-bbe5-fdcc8108abbd/discovery/0.log" Jul 30 03:26:05.961407 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:05.961378 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-z6xs4_d61232e1-244c-407e-9342-f6ed6bf47758/payload-pre-processing/0.log" Jul 30 03:26:07.959084 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:07.959059 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-xsv25_c1b537ec-1ee0-4ac5-ae12-cf7d38acad6f/migrator/0.log" Jul 30 03:26:07.978593 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:07.978567 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-xsv25_c1b537ec-1ee0-4ac5-ae12-cf7d38acad6f/graceful-termination/0.log" Jul 30 03:26:09.610686 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:09.610634 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fpm66_71aaec03-629a-427a-9207-6f58e2540c54/kube-multus-additional-cni-plugins/0.log" Jul 30 03:26:09.628386 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:09.628359 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fpm66_71aaec03-629a-427a-9207-6f58e2540c54/egress-router-binary-copy/0.log" Jul 30 03:26:09.648454 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:09.648437 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fpm66_71aaec03-629a-427a-9207-6f58e2540c54/cni-plugins/0.log" Jul 30 03:26:09.665693 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:09.665675 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fpm66_71aaec03-629a-427a-9207-6f58e2540c54/bond-cni-plugin/0.log" Jul 30 03:26:09.691518 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:09.691499 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fpm66_71aaec03-629a-427a-9207-6f58e2540c54/routeoverride-cni/0.log" Jul 30 03:26:09.709631 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:09.709614 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fpm66_71aaec03-629a-427a-9207-6f58e2540c54/whereabouts-cni-bincopy/0.log" Jul 30 03:26:09.727917 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:09.727897 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fpm66_71aaec03-629a-427a-9207-6f58e2540c54/whereabouts-cni/0.log" Jul 30 03:26:09.757423 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:09.757403 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gmdnj_78492ecc-d43f-4513-a1f7-0470faef225f/kube-multus/0.log" Jul 30 03:26:09.811541 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:09.811520 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rx95s_db23d664-ec37-4206-b1f3-ac52ecec65dc/network-metrics-daemon/0.log" Jul 30 03:26:09.827130 ip-10-0-131-5 kubenswrapper[2566]: I0730 03:26:09.827112 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rx95s_db23d664-ec37-4206-b1f3-ac52ecec65dc/kube-rbac-proxy/0.log"