Sep 07 12:52:45.862269 ip-10-0-142-1 systemd[1]: Starting Kubernetes Kubelet... Sep 07 12:52:46.284154 ip-10-0-142-1 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 12:52:46.284154 ip-10-0-142-1 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 07 12:52:46.284154 ip-10-0-142-1 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 12:52:46.284731 ip-10-0-142-1 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 07 12:52:46.284731 ip-10-0-142-1 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 12:52:46.285143 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.285067 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 07 12:52:46.289288 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289275 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 12:52:46.289288 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289288 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289292 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289295 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289298 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289302 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289305 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289308 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289311 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289318 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289321 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289324 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289326 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289329 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289331 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289334 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289337 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289339 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289342 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289344 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289346 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 12:52:46.289349 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289349 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289352 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289354 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289357 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289360 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289363 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289366 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289368 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289371 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289374 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289376 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289379 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289381 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289383 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289386 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289388 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289391 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289393 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289396 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 12:52:46.289826 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289398 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289400 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289403 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289405 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289407 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289409 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289412 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289414 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289416 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289419 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289421 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289424 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289427 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289429 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289431 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289436 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289440 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289443 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289445 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 12:52:46.290519 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289448 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289450 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289453 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289455 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289458 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289461 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289465 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289467 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289471 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289473 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289476 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289478 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289481 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289483 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289486 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289489 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289491 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289494 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289496 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289498 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 07 12:52:46.291026 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289501 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289504 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289507 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289510 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289512 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289515 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289518 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289520 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289523 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289526 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289528 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289531 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289535 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289538 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289541 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289543 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289546 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289549 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289552 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 12:52:46.291635 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.289554 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291923 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291934 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291938 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291941 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291944 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291946 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291949 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291953 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291955 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291958 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291961 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291963 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291980 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291983 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291985 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291988 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291990 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291993 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291995 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 12:52:46.292116 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.291998 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292000 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292003 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292005 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292009 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292011 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292014 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292017 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292021 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292025 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292028 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292030 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292033 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292036 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292039 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292042 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292044 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292047 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292050 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 12:52:46.292608 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292052 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292055 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292057 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292060 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292062 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292065 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292067 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292070 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292072 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292075 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292077 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292079 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292082 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292084 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292087 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292089 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292091 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292094 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292096 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 07 12:52:46.293095 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292098 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292102 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292105 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292108 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292111 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292114 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292116 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292119 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292123 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292127 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292129 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292132 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292136 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292139 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292142 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292145 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292148 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292150 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292153 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 12:52:46.293550 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292155 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292158 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292162 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292164 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292167 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292169 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292172 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292175 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292178 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292181 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292183 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292186 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292189 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292191 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292194 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292196 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292198 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292201 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292203 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292206 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 12:52:46.294036 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292208 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292212 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292215 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292556 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292561 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292563 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292566 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292569 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292571 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292574 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292576 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292579 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292581 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292584 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292586 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292589 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292591 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292594 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292596 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 12:52:46.294522 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292599 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292604 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292607 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292611 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292614 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292617 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292620 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292623 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292625 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292628 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292631 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292633 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292636 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292638 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292643 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292647 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292650 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292653 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 12:52:46.295010 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292655 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292659 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292662 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292664 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292666 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292669 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292671 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292674 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292676 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292678 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292681 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292683 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292685 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292688 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292690 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292694 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292696 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292698 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292701 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292704 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 12:52:46.295451 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292706 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292709 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292712 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292714 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292717 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292720 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292722 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292725 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292727 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292730 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292732 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292735 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292737 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292740 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292742 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292744 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292747 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292749 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292751 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292754 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 07 12:52:46.295916 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292756 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292758 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292761 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292763 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292765 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292768 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292770 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292772 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292775 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292779 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292782 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292785 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292788 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292790 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292793 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292796 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292798 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292800 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292803 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292805 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 12:52:46.296472 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292808 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292810 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292812 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292815 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.292817 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.292904 2576 flags.go:64] FLAG: --address="0.0.0.0" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.292949 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.292956 2576 flags.go:64] FLAG: --anonymous-auth="true" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.292961 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.292965 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.292986 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.292991 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.292995 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.292998 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293001 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293004 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293007 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293010 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293013 2576 flags.go:64] FLAG: --cgroup-root="" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293015 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293018 2576 flags.go:64] FLAG: --client-ca-file="" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293021 2576 flags.go:64] FLAG: --cloud-provider="external" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293024 2576 flags.go:64] FLAG: --cluster-dns="[]" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293028 2576 flags.go:64] FLAG: --cluster-domain="" Sep 07 12:52:46.296964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293031 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293034 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293037 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293040 2576 flags.go:64] FLAG: --container-log-max-files="5" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293048 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293051 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293054 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293057 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293060 2576 flags.go:64] FLAG: --contention-profiling="false" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293062 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293065 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293068 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293071 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293074 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293077 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293080 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293083 2576 flags.go:64] FLAG: --enable-load-reader="false" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293086 2576 flags.go:64] FLAG: --enable-server="true" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293089 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293093 2576 flags.go:64] FLAG: --event-burst="100" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293096 2576 flags.go:64] FLAG: --event-qps="50" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293098 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293101 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293104 2576 flags.go:64] FLAG: --eviction-hard="" Sep 07 12:52:46.297574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293107 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293110 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293113 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293116 2576 flags.go:64] FLAG: --eviction-soft="" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293119 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293121 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293126 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293129 2576 flags.go:64] FLAG: --experimental-mounter-path="" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293131 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293134 2576 flags.go:64] FLAG: --fail-swap-on="true" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293137 2576 flags.go:64] FLAG: --feature-gates="" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293140 2576 flags.go:64] FLAG: --file-check-frequency="20s" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293143 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293146 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293149 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293151 2576 flags.go:64] FLAG: --healthz-port="10248" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293154 2576 flags.go:64] FLAG: --help="false" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293157 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293160 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293162 2576 flags.go:64] FLAG: --http-check-frequency="20s" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293165 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293168 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293171 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293174 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 07 12:52:46.298164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293177 2576 flags.go:64] FLAG: --image-service-endpoint="" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293179 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293182 2576 flags.go:64] FLAG: --kube-api-burst="100" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293184 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293187 2576 flags.go:64] FLAG: --kube-api-qps="50" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293190 2576 flags.go:64] FLAG: --kube-reserved="" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293193 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293195 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293198 2576 flags.go:64] FLAG: --kubelet-cgroups="" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293200 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293203 2576 flags.go:64] FLAG: --lock-file="" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293205 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293208 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293211 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293216 2576 flags.go:64] FLAG: --log-json-split-stream="false" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293219 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293222 2576 flags.go:64] FLAG: --log-text-split-stream="false" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293225 2576 flags.go:64] FLAG: --logging-format="text" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293227 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293231 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293233 2576 flags.go:64] FLAG: --manifest-url="" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293236 2576 flags.go:64] FLAG: --manifest-url-header="" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293242 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293245 2576 flags.go:64] FLAG: --max-open-files="1000000" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293249 2576 flags.go:64] FLAG: --max-pods="110" Sep 07 12:52:46.298864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293251 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293254 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293257 2576 flags.go:64] FLAG: --memory-manager-policy="None" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293259 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293262 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293265 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293267 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293274 2576 flags.go:64] FLAG: --node-status-max-images="50" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293277 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293280 2576 flags.go:64] FLAG: --oom-score-adj="-999" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293282 2576 flags.go:64] FLAG: --pod-cidr="" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293285 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293290 2576 flags.go:64] FLAG: --pod-manifest-path="" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293293 2576 flags.go:64] FLAG: --pod-max-pids="-1" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293296 2576 flags.go:64] FLAG: --pods-per-core="0" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293298 2576 flags.go:64] FLAG: --port="10250" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293301 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293304 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f5fb660697b14755" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293307 2576 flags.go:64] FLAG: --qos-reserved="" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293310 2576 flags.go:64] FLAG: --read-only-port="10255" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293314 2576 flags.go:64] FLAG: --register-node="true" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293317 2576 flags.go:64] FLAG: --register-with-taints="" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293320 2576 flags.go:64] FLAG: --registry-burst="10" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293324 2576 flags.go:64] FLAG: --registry-qps="5" Sep 07 12:52:46.299496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293326 2576 flags.go:64] FLAG: --reserved-cpus="" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293330 2576 flags.go:64] FLAG: --reserved-memory="" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293333 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293337 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293339 2576 flags.go:64] FLAG: --rotate-certificates="false" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293342 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293345 2576 flags.go:64] FLAG: --runonce="false" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293348 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293351 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293354 2576 flags.go:64] FLAG: --seccomp-default="false" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293356 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293359 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293362 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293365 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293368 2576 flags.go:64] FLAG: --storage-driver-password="root" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293370 2576 flags.go:64] FLAG: --storage-driver-secure="false" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293373 2576 flags.go:64] FLAG: --storage-driver-table="stats" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293376 2576 flags.go:64] FLAG: --storage-driver-user="root" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293378 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293381 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293384 2576 flags.go:64] FLAG: --system-cgroups="" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293386 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293392 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293395 2576 flags.go:64] FLAG: --tls-cert-file="" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293397 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 07 12:52:46.300079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293401 2576 flags.go:64] FLAG: --tls-min-version="" Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293404 2576 flags.go:64] FLAG: --tls-private-key-file="" Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293406 2576 flags.go:64] FLAG: --topology-manager-policy="none" Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293410 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293413 2576 flags.go:64] FLAG: --topology-manager-scope="container" Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293416 2576 flags.go:64] FLAG: --v="2" Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293419 2576 flags.go:64] FLAG: --version="false" Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293424 2576 flags.go:64] FLAG: --vmodule="" Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293429 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.293432 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293544 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293547 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293551 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293554 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293557 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293560 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293564 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293568 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293571 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293574 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293577 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 12:52:46.300694 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293580 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293582 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293584 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293587 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293589 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293592 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293594 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293596 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293599 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293601 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293604 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293606 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293608 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293611 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293615 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293617 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293620 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293622 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293624 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 12:52:46.301261 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293629 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293631 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293634 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293636 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293639 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293642 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293644 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293647 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293649 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293652 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293654 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293656 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293659 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293661 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293664 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293666 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293673 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293676 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293678 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293681 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 12:52:46.301718 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293683 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293686 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293688 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293690 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293693 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293695 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293698 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293701 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293704 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293707 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293709 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293711 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293715 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293718 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293720 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293723 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293725 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293728 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293731 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 12:52:46.302252 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293733 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293736 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293738 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293741 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293743 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293745 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293748 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293750 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293753 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293757 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293760 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293763 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293766 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293769 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293771 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293774 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293776 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293779 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293782 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 12:52:46.302726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293785 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293789 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293792 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293795 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293798 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293801 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293805 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293807 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293810 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293813 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.293815 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.294498 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.302161 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.302174 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.302186 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 07 12:52:46.303204 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.302191 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302242 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302247 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302250 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302253 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302256 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302259 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302261 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302264 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302267 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302270 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302272 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302275 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302278 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302280 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302283 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302285 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302288 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302290 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302295 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 12:52:46.303575 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302299 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302302 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302305 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302307 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302310 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302313 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302317 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302320 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302323 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302326 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302329 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302332 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302334 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302337 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302340 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302342 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302344 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302347 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302349 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 12:52:46.304072 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302352 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302354 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302357 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302359 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302362 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302366 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302369 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302372 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302375 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302377 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302380 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302383 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302385 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302387 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302390 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302392 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302395 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302397 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302399 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302402 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 12:52:46.304560 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302405 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302408 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302410 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302413 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302416 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302419 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302421 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302424 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302426 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302429 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302431 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302434 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302436 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302438 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302441 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302444 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302446 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302448 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302451 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 12:52:46.305066 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302453 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302456 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302458 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302461 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302463 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302466 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302468 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302470 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302473 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302475 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302478 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302480 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302483 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302485 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302488 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302491 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302494 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302497 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302499 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302502 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 12:52:46.305524 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302504 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302506 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.302511 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302643 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302649 2576 feature_gate.go:328] unrecognized feature gate: Example Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302652 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302654 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302657 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302660 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302662 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302664 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302667 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302669 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302672 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302674 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 12:52:46.306131 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302677 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302680 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302682 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302684 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302687 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302689 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302692 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302694 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302696 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302699 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302702 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302705 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302708 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302713 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302716 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302718 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302720 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302723 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302725 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302728 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 12:52:46.306492 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302731 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302733 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302736 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302739 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302741 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302744 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302746 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302749 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302751 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302753 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302756 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302758 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302761 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302764 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302766 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302769 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302771 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302773 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302776 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302778 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 12:52:46.306960 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302781 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302784 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302786 2576 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302789 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302792 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302794 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302797 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302801 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302804 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302807 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302810 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302813 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302815 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302818 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302821 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302823 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302826 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302828 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302830 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 12:52:46.307462 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302834 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302837 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302840 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302843 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302846 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302848 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302851 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302854 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302856 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302858 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302861 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302863 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302866 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302869 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302871 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302874 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302876 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302878 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302881 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302884 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 12:52:46.307919 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302886 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 12:52:46.308412 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302888 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 12:52:46.308412 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302891 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 12:52:46.308412 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302893 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 12:52:46.308412 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302895 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 12:52:46.308412 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302898 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 12:52:46.308412 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302900 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 12:52:46.308412 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:46.302903 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 12:52:46.308412 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.302907 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 12:52:46.308412 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.303010 2576 server.go:962] "Client rotation is on, will bootstrap in background" Sep 07 12:52:46.308412 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.305294 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 07 12:52:46.308412 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.306383 2576 server.go:1019] "Starting client certificate rotation" Sep 07 12:52:46.308412 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.306489 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 07 12:52:46.308412 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.306521 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 07 12:52:46.309363 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.309350 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 07 12:52:46.311033 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.311017 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 07 12:52:46.323146 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.323130 2576 log.go:25] "Validated CRI v1 runtime API" Sep 07 12:52:46.327235 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.327222 2576 log.go:25] "Validated CRI v1 image API" Sep 07 12:52:46.328104 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.328090 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 07 12:52:46.330620 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.330593 2576 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c714406a-8214-4258-b508-0a33686dc717:/dev/nvme0n1p4 eb39eebc-7905-4c04-91bc-1ab1187f7313:/dev/nvme0n1p3] Sep 07 12:52:46.330704 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.330613 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 07 12:52:46.335718 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.335605 2576 manager.go:216] Machine: {Timestamp:2026-09-07 12:52:46.334169088 +0000 UTC m=+0.364918870 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100430 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2314a06d3ef9774d305c34cb77b129 SystemUUID:ec2314a0-6d3e-f977-4d30-5c34cb77b129 BootID:a784199a-8180-4858-933a-c222c2118b90 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4a:24:69:68:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4a:24:69:68:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:64:f6:75:db:06 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 07 12:52:46.335718 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.335707 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 07 12:52:46.335874 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.335805 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 07 12:52:46.339295 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.339267 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 07 12:52:46.339423 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.339297 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-1.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 07 12:52:46.340763 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.340753 2576 topology_manager.go:138] "Creating topology manager with none policy" Sep 07 12:52:46.340802 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.340764 2576 container_manager_linux.go:309] "Creating device plugin manager" Sep 07 12:52:46.340802 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.340772 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 07 12:52:46.341619 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.341608 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 07 12:52:46.343546 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.343534 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 07 12:52:46.345954 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.345937 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 07 12:52:46.349070 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.349057 2576 state_mem.go:36] "Initialized new in-memory state store" Sep 07 12:52:46.349195 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.349185 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 07 12:52:46.349231 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.349215 2576 kubelet.go:486] "Attempting to sync node with API server" Sep 07 12:52:46.349231 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.349225 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 07 12:52:46.349284 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.349250 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 07 12:52:46.349284 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.349269 2576 kubelet.go:398] "Adding apiserver pod source" Sep 07 12:52:46.349284 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.349279 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 07 12:52:46.349646 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.349632 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 07 12:52:46.349704 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.349650 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 07 12:52:46.352090 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.352076 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 07 12:52:46.355312 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.355296 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 07 12:52:46.355387 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.355335 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 07 12:52:46.355534 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.355519 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 07 12:52:46.355601 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.355546 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 07 12:52:46.355601 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.355560 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 07 12:52:46.355601 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.355568 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 07 12:52:46.355601 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.355578 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 07 12:52:46.355601 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.355587 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 07 12:52:46.356193 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.356181 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 07 12:52:46.356241 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.356195 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 07 12:52:46.356241 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.356207 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 07 12:52:46.356241 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.356215 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 07 12:52:46.356241 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.356234 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 07 12:52:46.356375 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.356246 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 07 12:52:46.357167 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.357156 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 07 12:52:46.357228 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.357170 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 07 12:52:46.360779 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.360763 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-1.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 12:52:46.361109 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.361097 2576 server.go:1268] "Started kubelet" Sep 07 12:52:46.365390 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.365350 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 07 12:52:46.365499 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.361157 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 07 12:52:46.366076 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.365708 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 07 12:52:46.366076 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.365777 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 07 12:52:46.366076 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.365783 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-1.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 07 12:52:46.366116 ip-10-0-142-1 systemd[1]: Started Kubernetes Kubelet. Sep 07 12:52:46.367667 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.367536 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 07 12:52:46.368099 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.368078 2576 server.go:310] "Adding debug handlers to kubelet server" Sep 07 12:52:46.370432 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.370414 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 07 12:52:46.372625 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.371596 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-1.ec2.internal.18d30afec64c2d49 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-1.ec2.internal,UID:ip-10-0-142-1.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-1.ec2.internal,},FirstTimestamp:2026-09-07 12:52:46.361070921 +0000 UTC m=+0.391820684,LastTimestamp:2026-09-07 12:52:46.361070921 +0000 UTC m=+0.391820684,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-1.ec2.internal,}" Sep 07 12:52:46.374371 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.374357 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 07 12:52:46.374429 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.374369 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 07 12:52:46.374553 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.374534 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-1.ec2.internal\" not found" Sep 07 12:52:46.375241 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.375082 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 07 12:52:46.375241 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.375243 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 07 12:52:46.375390 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.375081 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 07 12:52:46.375390 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.375302 2576 factory.go:153] Registering CRI-O factory Sep 07 12:52:46.375390 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.375338 2576 factory.go:223] Registration of the crio container factory successfully Sep 07 12:52:46.375390 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.375349 2576 reconstruct.go:97] "Volume reconstruction finished" Sep 07 12:52:46.375390 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.375357 2576 reconciler.go:29] "Reconciler: start to sync state" Sep 07 12:52:46.375569 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.375398 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 07 12:52:46.375569 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.375406 2576 factory.go:55] Registering systemd factory Sep 07 12:52:46.375569 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.375412 2576 factory.go:223] Registration of the systemd container factory successfully Sep 07 12:52:46.375569 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.375427 2576 factory.go:103] Registering Raw factory Sep 07 12:52:46.375569 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.375439 2576 manager.go:1196] Started watching for new ooms in manager Sep 07 12:52:46.375823 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.375812 2576 manager.go:349] Starting recovery of all containers Sep 07 12:52:46.382854 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.382828 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-1.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 07 12:52:46.382947 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.382911 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 07 12:52:46.384545 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.384530 2576 manager.go:354] Recovery completed Sep 07 12:52:46.389986 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.389953 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 12:52:46.391647 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.391632 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientMemory" Sep 07 12:52:46.392066 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.392052 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 12:52:46.392134 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.392071 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientPID" Sep 07 12:52:46.392585 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.392573 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 07 12:52:46.392585 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.392583 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 07 12:52:46.392664 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.392585 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 12:52:46.392711 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.392597 2576 state_mem.go:36] "Initialized new in-memory state store" Sep 07 12:52:46.393250 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.393237 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientMemory" Sep 07 12:52:46.393324 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.393264 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 12:52:46.393324 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.393274 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientPID" Sep 07 12:52:46.393939 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.393884 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-1.ec2.internal.18d30afec824ab80 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-1.ec2.internal,UID:ip-10-0-142-1.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-1.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-1.ec2.internal,},FirstTimestamp:2026-09-07 12:52:46.392036224 +0000 UTC m=+0.422785985,LastTimestamp:2026-09-07 12:52:46.392036224 +0000 UTC m=+0.422785985,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-1.ec2.internal,}" Sep 07 12:52:46.394805 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.394792 2576 policy_none.go:49] "None policy: Start" Sep 07 12:52:46.394850 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.394809 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 07 12:52:46.394850 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.394819 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 07 12:52:46.396791 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.396780 2576 policy_none.go:47] "Start" Sep 07 12:52:46.401378 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.401350 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2jcgb" Sep 07 12:52:46.403267 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.403203 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-1.ec2.internal.18d30afec82510ec default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-1.ec2.internal,UID:ip-10-0-142-1.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-1.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-1.ec2.internal,},FirstTimestamp:2026-09-07 12:52:46.392062188 +0000 UTC m=+0.422811954,LastTimestamp:2026-09-07 12:52:46.392062188 +0000 UTC m=+0.422811954,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-1.ec2.internal,}" Sep 07 12:52:46.410366 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.410347 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2jcgb" Sep 07 12:52:46.414861 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.414777 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-1.ec2.internal.18d30afec8254b75 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-1.ec2.internal,UID:ip-10-0-142-1.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-1.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-1.ec2.internal,},FirstTimestamp:2026-09-07 12:52:46.392077173 +0000 UTC m=+0.422826936,LastTimestamp:2026-09-07 12:52:46.392077173 +0000 UTC m=+0.422826936,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-1.ec2.internal,}" Sep 07 12:52:46.432617 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.432600 2576 manager.go:337] "Starting Device Plugin manager" Sep 07 12:52:46.454832 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.432635 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 07 12:52:46.454832 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.432647 2576 server.go:87] "Starting device plugin registration server" Sep 07 12:52:46.454832 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.432939 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 07 12:52:46.454832 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.432949 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 07 12:52:46.454832 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.433062 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 07 12:52:46.454832 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.433143 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 07 12:52:46.454832 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.433152 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 07 12:52:46.454832 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.433647 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 07 12:52:46.454832 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.433702 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-1.ec2.internal\" not found" Sep 07 12:52:46.474328 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.474297 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 07 12:52:46.475558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.475544 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 07 12:52:46.475625 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.475589 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 07 12:52:46.475625 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.475611 2576 kubelet.go:2448] "Starting kubelet main sync loop" Sep 07 12:52:46.475695 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.475644 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 07 12:52:46.478341 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.478313 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 12:52:46.533629 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.533520 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 12:52:46.534528 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.534510 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientMemory" Sep 07 12:52:46.534622 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.534539 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 12:52:46.534622 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.534551 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientPID" Sep 07 12:52:46.534622 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.534574 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.544615 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.544601 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.544674 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.544623 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-1.ec2.internal\": node \"ip-10-0-142-1.ec2.internal\" not found" Sep 07 12:52:46.575957 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.575929 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-1.ec2.internal"] Sep 07 12:52:46.576063 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.576013 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 12:52:46.577404 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.577390 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientMemory" Sep 07 12:52:46.577468 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.577418 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 12:52:46.577468 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.577433 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientPID" Sep 07 12:52:46.579795 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.579781 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 12:52:46.579941 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.579927 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.580042 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.579953 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 12:52:46.580490 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.580473 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientMemory" Sep 07 12:52:46.580557 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.580502 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 12:52:46.580557 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.580513 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientPID" Sep 07 12:52:46.580557 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.580525 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientMemory" Sep 07 12:52:46.580557 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.580548 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 12:52:46.580557 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.580557 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientPID" Sep 07 12:52:46.582811 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.582798 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.582864 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.582822 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 12:52:46.583568 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.583554 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientMemory" Sep 07 12:52:46.583630 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.583577 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 12:52:46.583630 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.583586 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeHasSufficientPID" Sep 07 12:52:46.584132 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.584116 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-1.ec2.internal\" not found" Sep 07 12:52:46.596533 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.596513 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-1.ec2.internal\" not found" node="ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.600333 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.600318 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-1.ec2.internal\" not found" node="ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.684795 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.684772 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-1.ec2.internal\" not found" Sep 07 12:52:46.777075 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.777034 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4351a43c17868aab7e62d03e90366ae5-config\") pod \"kube-apiserver-proxy-ip-10-0-142-1.ec2.internal\" (UID: \"4351a43c17868aab7e62d03e90366ae5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.777211 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.777089 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a94a700a9764e0ad82bdf73a6f15aedb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal\" (UID: \"a94a700a9764e0ad82bdf73a6f15aedb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.777211 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.777110 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a94a700a9764e0ad82bdf73a6f15aedb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal\" (UID: \"a94a700a9764e0ad82bdf73a6f15aedb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.785163 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.785113 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-1.ec2.internal\" not found" Sep 07 12:52:46.877777 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.877751 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4351a43c17868aab7e62d03e90366ae5-config\") pod \"kube-apiserver-proxy-ip-10-0-142-1.ec2.internal\" (UID: \"4351a43c17868aab7e62d03e90366ae5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.877865 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.877779 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a94a700a9764e0ad82bdf73a6f15aedb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal\" (UID: \"a94a700a9764e0ad82bdf73a6f15aedb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.877865 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.877795 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a94a700a9764e0ad82bdf73a6f15aedb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal\" (UID: \"a94a700a9764e0ad82bdf73a6f15aedb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.877865 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.877835 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a94a700a9764e0ad82bdf73a6f15aedb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal\" (UID: \"a94a700a9764e0ad82bdf73a6f15aedb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.878020 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.877867 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4351a43c17868aab7e62d03e90366ae5-config\") pod \"kube-apiserver-proxy-ip-10-0-142-1.ec2.internal\" (UID: \"4351a43c17868aab7e62d03e90366ae5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.878020 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.877891 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a94a700a9764e0ad82bdf73a6f15aedb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal\" (UID: \"a94a700a9764e0ad82bdf73a6f15aedb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.885866 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.885852 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-1.ec2.internal\" not found" Sep 07 12:52:46.900025 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.900007 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.903121 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:46.903106 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-1.ec2.internal" Sep 07 12:52:46.986648 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:46.986615 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-1.ec2.internal\" not found" Sep 07 12:52:47.087216 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.087144 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-1.ec2.internal\" not found" Sep 07 12:52:47.187645 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.187616 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-1.ec2.internal\" not found" Sep 07 12:52:47.197501 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.197477 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 12:52:47.274561 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.274532 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal" Sep 07 12:52:47.290169 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.290147 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 07 12:52:47.291763 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.291744 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-1.ec2.internal" Sep 07 12:52:47.300610 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.300594 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 07 12:52:47.305696 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.305683 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 07 12:52:47.305801 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.305784 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 07 12:52:47.305841 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.305814 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 07 12:52:47.350141 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.350070 2576 apiserver.go:52] "Watching apiserver" Sep 07 12:52:47.366315 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.366291 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 07 12:52:47.367825 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.367802 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-hpq9f","kube-system/konnectivity-agent-vnn5z","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv","openshift-dns/node-resolver-2lm8h","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal","openshift-multus/network-metrics-daemon-pkv6v","openshift-network-diagnostics/network-check-target-m9c7v","openshift-network-operator/iptables-alerter-nlq4q","kube-system/kube-apiserver-proxy-ip-10-0-142-1.ec2.internal","openshift-cluster-node-tuning-operator/tuned-t8qbk","openshift-image-registry/node-ca-mmqsg","openshift-multus/multus-additional-cni-plugins-6dj5g","openshift-multus/multus-vgpxq","openshift-ovn-kubernetes/ovnkube-node-lpc6s"] Sep 07 12:52:47.370073 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.370056 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:47.370161 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.370131 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:52:47.373077 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.373049 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vnn5z" Sep 07 12:52:47.374522 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.374506 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 07 12:52:47.374854 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.374806 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 07 12:52:47.374947 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.374870 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 07 12:52:47.374947 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.374891 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-95wsq\"" Sep 07 12:52:47.375423 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.375407 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.376882 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.376853 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 07 12:52:47.377007 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.376924 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 07 12:52:47.377007 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.376965 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x4x5k\"" Sep 07 12:52:47.377132 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.377012 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 07 12:52:47.377492 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.377478 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2lm8h" Sep 07 12:52:47.379584 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.379567 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:47.379663 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.379638 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:52:47.379785 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.379767 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7296228-d10d-43e8-9c48-d6d72e58532a-tmp-dir\") pod \"node-resolver-2lm8h\" (UID: \"a7296228-d10d-43e8-9c48-d6d72e58532a\") " pod="openshift-dns/node-resolver-2lm8h" Sep 07 12:52:47.379889 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.379800 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7vbg\" (UniqueName: \"kubernetes.io/projected/a7296228-d10d-43e8-9c48-d6d72e58532a-kube-api-access-z7vbg\") pod \"node-resolver-2lm8h\" (UID: \"a7296228-d10d-43e8-9c48-d6d72e58532a\") " pod="openshift-dns/node-resolver-2lm8h" Sep 07 12:52:47.379889 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.379824 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:47.379889 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.379863 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-device-dir\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.379889 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.379882 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-etc-selinux\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.380097 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.379897 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ph4j\" (UniqueName: \"kubernetes.io/projected/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-kube-api-access-4ph4j\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.380097 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.379948 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a7296228-d10d-43e8-9c48-d6d72e58532a-hosts-file\") pod \"node-resolver-2lm8h\" (UID: \"a7296228-d10d-43e8-9c48-d6d72e58532a\") " pod="openshift-dns/node-resolver-2lm8h" Sep 07 12:52:47.380097 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.379995 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-dbus\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:47.380097 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.380013 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-registration-dir\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.380097 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.380027 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-sys-fs\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.380097 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.380041 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-kubelet-config\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:47.380097 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.380079 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.380353 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.380102 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-socket-dir\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.380353 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.380119 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8c872390-d8c4-46eb-9799-cbde30173723-agent-certs\") pod \"konnectivity-agent-vnn5z\" (UID: \"8c872390-d8c4-46eb-9799-cbde30173723\") " pod="kube-system/konnectivity-agent-vnn5z" Sep 07 12:52:47.380353 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.380144 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8c872390-d8c4-46eb-9799-cbde30173723-konnectivity-ca\") pod \"konnectivity-agent-vnn5z\" (UID: \"8c872390-d8c4-46eb-9799-cbde30173723\") " pod="kube-system/konnectivity-agent-vnn5z" Sep 07 12:52:47.380511 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.380498 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 07 12:52:47.381065 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.381040 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 07 12:52:47.381127 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.381119 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wzwnr\"" Sep 07 12:52:47.381618 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.381604 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:52:47.381693 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.381674 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:52:47.384379 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.384367 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nlq4q" Sep 07 12:52:47.385763 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.385712 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 07 12:52:47.385842 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.385780 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4t7d6\"" Sep 07 12:52:47.385903 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.385854 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 07 12:52:47.386009 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.385920 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 07 12:52:47.386185 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.386169 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 07 12:52:47.386581 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.386567 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.388611 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.388593 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mmqsg" Sep 07 12:52:47.388684 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.388642 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 07 12:52:47.388684 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.388675 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 07 12:52:47.389031 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.389016 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xrfs8\"" Sep 07 12:52:47.390104 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.390087 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8qtfh\"" Sep 07 12:52:47.390196 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.390114 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 07 12:52:47.390312 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.390298 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 07 12:52:47.391033 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.391013 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.391120 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.391049 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 07 12:52:47.392853 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.392836 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 07 12:52:47.392990 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.392923 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 07 12:52:47.392990 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.392944 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 07 12:52:47.392990 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.392946 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 07 12:52:47.393366 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.393340 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 07 12:52:47.393366 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.393353 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fjwcw\"" Sep 07 12:52:47.393527 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.393506 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.395169 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.395152 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 07 12:52:47.395766 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.395152 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sqq26\"" Sep 07 12:52:47.395766 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.395706 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.397110 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.397089 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 07 12:52:47.397110 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.397107 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 07 12:52:47.397790 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.397771 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 07 12:52:47.397915 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.397820 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wr69z\"" Sep 07 12:52:47.397915 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.397825 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 07 12:52:47.398060 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.397912 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 07 12:52:47.398060 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.397933 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 07 12:52:47.412282 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.412259 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bflfk" Sep 07 12:52:47.413505 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.413456 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 12:47:46 +0000 UTC" deadline="2028-02-27 16:08:06.359658408 +0000 UTC" Sep 07 12:52:47.413505 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.413502 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12915h15m18.946158807s" Sep 07 12:52:47.421011 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.420961 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bflfk" Sep 07 12:52:47.438799 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:47.438770 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4351a43c17868aab7e62d03e90366ae5.slice/crio-6f57ce4b29767c1ec18bcfc3c6aa75fe10cfb0d3f9ee713b421f6a6423833065 WatchSource:0}: Error finding container 6f57ce4b29767c1ec18bcfc3c6aa75fe10cfb0d3f9ee713b421f6a6423833065: Status 404 returned error can't find the container with id 6f57ce4b29767c1ec18bcfc3c6aa75fe10cfb0d3f9ee713b421f6a6423833065 Sep 07 12:52:47.443173 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.443156 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 12:52:47.476497 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.476482 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 07 12:52:47.478358 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.478325 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-1.ec2.internal" event={"ID":"4351a43c17868aab7e62d03e90366ae5","Type":"ContainerStarted","Data":"6f57ce4b29767c1ec18bcfc3c6aa75fe10cfb0d3f9ee713b421f6a6423833065"} Sep 07 12:52:47.480588 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480571 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-sys\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.480646 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480596 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs\") pod \"network-metrics-daemon-pkv6v\" (UID: \"da204f77-d893-4332-baab-4418cb7e72bc\") " pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:47.480646 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480612 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-system-cni-dir\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.480646 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480625 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-log-socket\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.480763 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480677 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-847cr\" (UniqueName: \"kubernetes.io/projected/da204f77-d893-4332-baab-4418cb7e72bc-kube-api-access-847cr\") pod \"network-metrics-daemon-pkv6v\" (UID: \"da204f77-d893-4332-baab-4418cb7e72bc\") " pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:47.480763 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480697 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-kubelet\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.480763 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480714 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-systemd-units\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.480763 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480728 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-cni-bin\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.480763 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480746 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-kubelet-config\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:47.480763 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480760 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvzvs\" (UniqueName: \"kubernetes.io/projected/67ad982a-f807-4a65-b78d-8427d008e9cd-kube-api-access-hvzvs\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.480995 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480776 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a5430f6-3492-4a63-8126-d2080e0dcc3c-cni-binary-copy\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.480995 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480792 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtvwt\" (UniqueName: \"kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt\") pod \"network-check-target-m9c7v\" (UID: \"7b8afbb2-9557-4821-8712-89cc16f472f4\") " pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:52:47.480995 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480804 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-kubelet-config\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:47.480995 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480811 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-run-systemd\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.480995 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480841 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f87025ad-1367-41bd-a774-3f27dc223933-ovn-node-metrics-cert\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.480995 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480862 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-lib-modules\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.480995 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480886 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/67ad982a-f807-4a65-b78d-8427d008e9cd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.480995 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480914 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-etc-openvswitch\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.480995 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480937 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.480995 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.480986 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481008 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-etc-selinux\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481023 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ph4j\" (UniqueName: \"kubernetes.io/projected/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-kube-api-access-4ph4j\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481037 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-tuned\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.481046 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481050 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-cnibin\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481089 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-run-k8s-cni-cncf-io\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.481121 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret podName:6e766773-bcc2-4cb6-8df6-3dc08e9d59cc nodeName:}" failed. No retries permitted until 2026-09-07 12:52:47.981095861 +0000 UTC m=+2.011845610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret") pod "global-pull-secret-syncer-hpq9f" (UID: "6e766773-bcc2-4cb6-8df6-3dc08e9d59cc") : object "kube-system"/"original-pull-secret" not registered Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481153 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-etc-selinux\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481178 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a5430f6-3492-4a63-8126-d2080e0dcc3c-multus-daemon-config\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481211 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-dbus\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481243 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-sys-fs\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481253 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-dbus\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481259 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67ad982a-f807-4a65-b78d-8427d008e9cd-cnibin\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481285 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-run-multus-certs\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481306 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmmhs\" (UniqueName: \"kubernetes.io/projected/7344905a-1586-4418-b3af-c26fee920038-kube-api-access-wmmhs\") pod \"iptables-alerter-nlq4q\" (UID: \"7344905a-1586-4418-b3af-c26fee920038\") " pod="openshift-network-operator/iptables-alerter-nlq4q" Sep 07 12:52:47.481356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481314 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-sys-fs\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481327 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481342 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-sysctl-d\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481356 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-systemd\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481375 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-var-lib-kubelet\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481405 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-multus-cni-dir\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481416 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481426 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-var-lib-cni-bin\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481453 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f87025ad-1367-41bd-a774-3f27dc223933-ovnkube-config\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481475 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-kubernetes\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481497 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-run\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481515 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-var-lib-cni-multus\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481537 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-multus-conf-dir\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481559 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f87025ad-1367-41bd-a774-3f27dc223933-ovnkube-script-lib\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481575 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-sysctl-conf\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481611 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7296228-d10d-43e8-9c48-d6d72e58532a-tmp-dir\") pod \"node-resolver-2lm8h\" (UID: \"a7296228-d10d-43e8-9c48-d6d72e58532a\") " pod="openshift-dns/node-resolver-2lm8h" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481629 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7vbg\" (UniqueName: \"kubernetes.io/projected/a7296228-d10d-43e8-9c48-d6d72e58532a-kube-api-access-z7vbg\") pod \"node-resolver-2lm8h\" (UID: \"a7296228-d10d-43e8-9c48-d6d72e58532a\") " pod="openshift-dns/node-resolver-2lm8h" Sep 07 12:52:47.481871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481643 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-host\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481656 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ad982a-f807-4a65-b78d-8427d008e9cd-system-cni-dir\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481673 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/67ad982a-f807-4a65-b78d-8427d008e9cd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481699 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-registration-dir\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481722 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67ad982a-f807-4a65-b78d-8427d008e9cd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481787 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-run-ovn-kubernetes\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481799 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-registration-dir\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481807 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f87025ad-1367-41bd-a774-3f27dc223933-env-overrides\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481830 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-socket-dir\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481845 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8840b2d7-5f57-4887-9d76-1966edbec149-host\") pod \"node-ca-mmqsg\" (UID: \"8840b2d7-5f57-4887-9d76-1966edbec149\") " pod="openshift-image-registry/node-ca-mmqsg" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481866 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-var-lib-kubelet\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481885 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-hostroot\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481906 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz7kg\" (UniqueName: \"kubernetes.io/projected/6a5430f6-3492-4a63-8126-d2080e0dcc3c-kube-api-access-xz7kg\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481927 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-socket-dir\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481934 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7344905a-1586-4418-b3af-c26fee920038-host-slash\") pod \"iptables-alerter-nlq4q\" (UID: \"7344905a-1586-4418-b3af-c26fee920038\") " pod="openshift-network-operator/iptables-alerter-nlq4q" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481956 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-run-netns\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.482558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.481993 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7296228-d10d-43e8-9c48-d6d72e58532a-tmp-dir\") pod \"node-resolver-2lm8h\" (UID: \"a7296228-d10d-43e8-9c48-d6d72e58532a\") " pod="openshift-dns/node-resolver-2lm8h" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482005 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-node-log\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482021 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8c872390-d8c4-46eb-9799-cbde30173723-konnectivity-ca\") pod \"konnectivity-agent-vnn5z\" (UID: \"8c872390-d8c4-46eb-9799-cbde30173723\") " pod="kube-system/konnectivity-agent-vnn5z" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482041 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-sysconfig\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482072 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dn4g\" (UniqueName: \"kubernetes.io/projected/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-kube-api-access-5dn4g\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482090 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhkfz\" (UniqueName: \"kubernetes.io/projected/8840b2d7-5f57-4887-9d76-1966edbec149-kube-api-access-mhkfz\") pod \"node-ca-mmqsg\" (UID: \"8840b2d7-5f57-4887-9d76-1966edbec149\") " pod="openshift-image-registry/node-ca-mmqsg" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482111 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-run-netns\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482127 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7344905a-1586-4418-b3af-c26fee920038-iptables-alerter-script\") pod \"iptables-alerter-nlq4q\" (UID: \"7344905a-1586-4418-b3af-c26fee920038\") " pod="openshift-network-operator/iptables-alerter-nlq4q" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482140 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-slash\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482152 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-run-openvswitch\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482178 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-device-dir\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482217 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-modprobe-d\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482225 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-device-dir\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482247 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8840b2d7-5f57-4887-9d76-1966edbec149-serviceca\") pod \"node-ca-mmqsg\" (UID: \"8840b2d7-5f57-4887-9d76-1966edbec149\") " pod="openshift-image-registry/node-ca-mmqsg" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482277 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67ad982a-f807-4a65-b78d-8427d008e9cd-cni-binary-copy\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482303 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-multus-socket-dir-parent\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.483017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482326 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72fj9\" (UniqueName: \"kubernetes.io/projected/f87025ad-1367-41bd-a774-3f27dc223933-kube-api-access-72fj9\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.483445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482353 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a7296228-d10d-43e8-9c48-d6d72e58532a-hosts-file\") pod \"node-resolver-2lm8h\" (UID: \"a7296228-d10d-43e8-9c48-d6d72e58532a\") " pod="openshift-dns/node-resolver-2lm8h" Sep 07 12:52:47.483445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482376 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-tmp\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.483445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482398 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-run-ovn\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.483445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482420 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-cni-netd\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.483445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482424 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a7296228-d10d-43e8-9c48-d6d72e58532a-hosts-file\") pod \"node-resolver-2lm8h\" (UID: \"a7296228-d10d-43e8-9c48-d6d72e58532a\") " pod="openshift-dns/node-resolver-2lm8h" Sep 07 12:52:47.483445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482442 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-var-lib-openvswitch\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.483445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482471 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8c872390-d8c4-46eb-9799-cbde30173723-agent-certs\") pod \"konnectivity-agent-vnn5z\" (UID: \"8c872390-d8c4-46eb-9799-cbde30173723\") " pod="kube-system/konnectivity-agent-vnn5z" Sep 07 12:52:47.483445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482479 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8c872390-d8c4-46eb-9799-cbde30173723-konnectivity-ca\") pod \"konnectivity-agent-vnn5z\" (UID: \"8c872390-d8c4-46eb-9799-cbde30173723\") " pod="kube-system/konnectivity-agent-vnn5z" Sep 07 12:52:47.483445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.482490 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-etc-kubernetes\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.483681 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.483651 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 07 12:52:47.487551 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.487440 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8c872390-d8c4-46eb-9799-cbde30173723-agent-certs\") pod \"konnectivity-agent-vnn5z\" (UID: \"8c872390-d8c4-46eb-9799-cbde30173723\") " pod="kube-system/konnectivity-agent-vnn5z" Sep 07 12:52:47.490152 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.490127 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7vbg\" (UniqueName: \"kubernetes.io/projected/a7296228-d10d-43e8-9c48-d6d72e58532a-kube-api-access-z7vbg\") pod \"node-resolver-2lm8h\" (UID: \"a7296228-d10d-43e8-9c48-d6d72e58532a\") " pod="openshift-dns/node-resolver-2lm8h" Sep 07 12:52:47.490216 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.490127 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ph4j\" (UniqueName: \"kubernetes.io/projected/d6d7f4fb-dd5a-4680-bfc1-b40129127fe7-kube-api-access-4ph4j\") pod \"aws-ebs-csi-driver-node-9vlkv\" (UID: \"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.525808 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.525788 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 12:52:47.583692 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583665 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-sysctl-conf\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.583692 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583694 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-host\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.583935 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583712 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ad982a-f807-4a65-b78d-8427d008e9cd-system-cni-dir\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.583935 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583728 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/67ad982a-f807-4a65-b78d-8427d008e9cd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.583935 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583748 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67ad982a-f807-4a65-b78d-8427d008e9cd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.583935 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583770 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-run-ovn-kubernetes\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.583935 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583792 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-host\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.583935 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583799 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ad982a-f807-4a65-b78d-8427d008e9cd-system-cni-dir\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.583935 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583853 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-run-ovn-kubernetes\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.583935 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583855 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-sysctl-conf\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.583935 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583886 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f87025ad-1367-41bd-a774-3f27dc223933-env-overrides\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.583935 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583912 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8840b2d7-5f57-4887-9d76-1966edbec149-host\") pod \"node-ca-mmqsg\" (UID: \"8840b2d7-5f57-4887-9d76-1966edbec149\") " pod="openshift-image-registry/node-ca-mmqsg" Sep 07 12:52:47.583935 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583929 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-var-lib-kubelet\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583929 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67ad982a-f807-4a65-b78d-8427d008e9cd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583951 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-hostroot\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583985 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8840b2d7-5f57-4887-9d76-1966edbec149-host\") pod \"node-ca-mmqsg\" (UID: \"8840b2d7-5f57-4887-9d76-1966edbec149\") " pod="openshift-image-registry/node-ca-mmqsg" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.583990 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-var-lib-kubelet\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584004 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-hostroot\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584005 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz7kg\" (UniqueName: \"kubernetes.io/projected/6a5430f6-3492-4a63-8126-d2080e0dcc3c-kube-api-access-xz7kg\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584088 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7344905a-1586-4418-b3af-c26fee920038-host-slash\") pod \"iptables-alerter-nlq4q\" (UID: \"7344905a-1586-4418-b3af-c26fee920038\") " pod="openshift-network-operator/iptables-alerter-nlq4q" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584107 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-run-netns\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584121 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-node-log\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584148 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-sysconfig\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584171 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7344905a-1586-4418-b3af-c26fee920038-host-slash\") pod \"iptables-alerter-nlq4q\" (UID: \"7344905a-1586-4418-b3af-c26fee920038\") " pod="openshift-network-operator/iptables-alerter-nlq4q" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584173 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dn4g\" (UniqueName: \"kubernetes.io/projected/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-kube-api-access-5dn4g\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584190 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-run-netns\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584206 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhkfz\" (UniqueName: \"kubernetes.io/projected/8840b2d7-5f57-4887-9d76-1966edbec149-kube-api-access-mhkfz\") pod \"node-ca-mmqsg\" (UID: \"8840b2d7-5f57-4887-9d76-1966edbec149\") " pod="openshift-image-registry/node-ca-mmqsg" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584227 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-sysconfig\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584231 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-run-netns\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584278 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-run-netns\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.584477 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584270 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7344905a-1586-4418-b3af-c26fee920038-iptables-alerter-script\") pod \"iptables-alerter-nlq4q\" (UID: \"7344905a-1586-4418-b3af-c26fee920038\") " pod="openshift-network-operator/iptables-alerter-nlq4q" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584297 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f87025ad-1367-41bd-a774-3f27dc223933-env-overrides\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584313 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-slash\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584336 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-run-openvswitch\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584347 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-slash\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584368 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-modprobe-d\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584388 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-run-openvswitch\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584395 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/67ad982a-f807-4a65-b78d-8427d008e9cd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584390 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8840b2d7-5f57-4887-9d76-1966edbec149-serviceca\") pod \"node-ca-mmqsg\" (UID: \"8840b2d7-5f57-4887-9d76-1966edbec149\") " pod="openshift-image-registry/node-ca-mmqsg" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584457 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67ad982a-f807-4a65-b78d-8427d008e9cd-cni-binary-copy\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584480 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-modprobe-d\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584484 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-multus-socket-dir-parent\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584508 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72fj9\" (UniqueName: \"kubernetes.io/projected/f87025ad-1367-41bd-a774-3f27dc223933-kube-api-access-72fj9\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584533 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-tmp\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584556 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-run-ovn\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584581 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-cni-netd\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584588 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-multus-socket-dir-parent\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.585514 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584603 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-var-lib-openvswitch\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584629 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-etc-kubernetes\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584630 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-cni-netd\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584653 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-sys\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584669 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-run-ovn\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584677 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs\") pod \"network-metrics-daemon-pkv6v\" (UID: \"da204f77-d893-4332-baab-4418cb7e72bc\") " pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584700 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-system-cni-dir\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584706 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-etc-kubernetes\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584723 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-log-socket\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584739 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-var-lib-openvswitch\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584758 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-847cr\" (UniqueName: \"kubernetes.io/projected/da204f77-d893-4332-baab-4418cb7e72bc-kube-api-access-847cr\") pod \"network-metrics-daemon-pkv6v\" (UID: \"da204f77-d893-4332-baab-4418cb7e72bc\") " pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.584779 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584785 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-sys\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584832 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7344905a-1586-4418-b3af-c26fee920038-iptables-alerter-script\") pod \"iptables-alerter-nlq4q\" (UID: \"7344905a-1586-4418-b3af-c26fee920038\") " pod="openshift-network-operator/iptables-alerter-nlq4q" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584828 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-kubelet\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584858 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8840b2d7-5f57-4887-9d76-1966edbec149-serviceca\") pod \"node-ca-mmqsg\" (UID: \"8840b2d7-5f57-4887-9d76-1966edbec149\") " pod="openshift-image-registry/node-ca-mmqsg" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584890 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-system-cni-dir\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584897 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-log-socket\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.586526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584932 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-node-log\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584938 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67ad982a-f807-4a65-b78d-8427d008e9cd-cni-binary-copy\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.584781 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-kubelet\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.584986 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs podName:da204f77-d893-4332-baab-4418cb7e72bc nodeName:}" failed. No retries permitted until 2026-09-07 12:52:48.084944149 +0000 UTC m=+2.115693899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs") pod "network-metrics-daemon-pkv6v" (UID: "da204f77-d893-4332-baab-4418cb7e72bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585015 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-systemd-units\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585075 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-systemd-units\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585101 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-cni-bin\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585127 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvzvs\" (UniqueName: \"kubernetes.io/projected/67ad982a-f807-4a65-b78d-8427d008e9cd-kube-api-access-hvzvs\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585166 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-cni-bin\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585169 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a5430f6-3492-4a63-8126-d2080e0dcc3c-cni-binary-copy\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585201 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtvwt\" (UniqueName: \"kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt\") pod \"network-check-target-m9c7v\" (UID: \"7b8afbb2-9557-4821-8712-89cc16f472f4\") " pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585239 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-run-systemd\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585278 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-run-systemd\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585380 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f87025ad-1367-41bd-a774-3f27dc223933-ovn-node-metrics-cert\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585419 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-lib-modules\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585436 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/67ad982a-f807-4a65-b78d-8427d008e9cd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.587445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585466 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-etc-openvswitch\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585491 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585531 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-tuned\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585555 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-lib-modules\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585586 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-cnibin\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585586 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-etc-openvswitch\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585610 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-run-k8s-cni-cncf-io\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585637 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a5430f6-3492-4a63-8126-d2080e0dcc3c-multus-daemon-config\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585653 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-cnibin\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585663 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67ad982a-f807-4a65-b78d-8427d008e9cd-cnibin\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585686 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-run-multus-certs\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585737 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-run-k8s-cni-cncf-io\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585735 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmmhs\" (UniqueName: \"kubernetes.io/projected/7344905a-1586-4418-b3af-c26fee920038-kube-api-access-wmmhs\") pod \"iptables-alerter-nlq4q\" (UID: \"7344905a-1586-4418-b3af-c26fee920038\") " pod="openshift-network-operator/iptables-alerter-nlq4q" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585800 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-sysctl-d\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585826 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-systemd\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585849 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-var-lib-kubelet\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585873 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-multus-cni-dir\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585898 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-var-lib-cni-bin\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585923 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f87025ad-1367-41bd-a774-3f27dc223933-ovnkube-config\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585946 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-kubernetes\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585954 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-run-multus-certs\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.585988 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-run\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586016 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-var-lib-cni-multus\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586040 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-multus-conf-dir\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586047 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/67ad982a-f807-4a65-b78d-8427d008e9cd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586062 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f87025ad-1367-41bd-a774-3f27dc223933-ovnkube-script-lib\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586108 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-systemd\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586170 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a5430f6-3492-4a63-8126-d2080e0dcc3c-multus-daemon-config\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586190 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-run\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586225 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67ad982a-f807-4a65-b78d-8427d008e9cd-cnibin\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586235 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-var-lib-cni-bin\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586232 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-sysctl-d\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586274 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-var-lib-kubelet\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586309 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-host-var-lib-cni-multus\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586330 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-kubernetes\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586361 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-multus-cni-dir\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.588645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586373 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a5430f6-3492-4a63-8126-d2080e0dcc3c-multus-conf-dir\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.589160 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586397 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f87025ad-1367-41bd-a774-3f27dc223933-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.589160 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586542 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f87025ad-1367-41bd-a774-3f27dc223933-ovnkube-script-lib\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.589160 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586643 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f87025ad-1367-41bd-a774-3f27dc223933-ovnkube-config\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.589160 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.586743 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a5430f6-3492-4a63-8126-d2080e0dcc3c-cni-binary-copy\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.589160 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.587362 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-tmp\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.589160 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.587780 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f87025ad-1367-41bd-a774-3f27dc223933-ovn-node-metrics-cert\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.589160 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.588054 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-etc-tuned\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.591503 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.591480 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 12:52:47.591503 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.591505 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 12:52:47.591681 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.591519 2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtvwt for pod openshift-network-diagnostics/network-check-target-m9c7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:52:47.591681 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.591571 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt podName:7b8afbb2-9557-4821-8712-89cc16f472f4 nodeName:}" failed. No retries permitted until 2026-09-07 12:52:48.091556686 +0000 UTC m=+2.122306457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wtvwt" (UniqueName: "kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt") pod "network-check-target-m9c7v" (UID: "7b8afbb2-9557-4821-8712-89cc16f472f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:52:47.593960 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.593929 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvzvs\" (UniqueName: \"kubernetes.io/projected/67ad982a-f807-4a65-b78d-8427d008e9cd-kube-api-access-hvzvs\") pod \"multus-additional-cni-plugins-6dj5g\" (UID: \"67ad982a-f807-4a65-b78d-8427d008e9cd\") " pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.594073 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.593934 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhkfz\" (UniqueName: \"kubernetes.io/projected/8840b2d7-5f57-4887-9d76-1966edbec149-kube-api-access-mhkfz\") pod \"node-ca-mmqsg\" (UID: \"8840b2d7-5f57-4887-9d76-1966edbec149\") " pod="openshift-image-registry/node-ca-mmqsg" Sep 07 12:52:47.594422 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.594398 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dn4g\" (UniqueName: \"kubernetes.io/projected/b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502-kube-api-access-5dn4g\") pod \"tuned-t8qbk\" (UID: \"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502\") " pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.594705 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.594687 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz7kg\" (UniqueName: \"kubernetes.io/projected/6a5430f6-3492-4a63-8126-d2080e0dcc3c-kube-api-access-xz7kg\") pod \"multus-vgpxq\" (UID: \"6a5430f6-3492-4a63-8126-d2080e0dcc3c\") " pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.594785 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.594758 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-847cr\" (UniqueName: \"kubernetes.io/projected/da204f77-d893-4332-baab-4418cb7e72bc-kube-api-access-847cr\") pod \"network-metrics-daemon-pkv6v\" (UID: \"da204f77-d893-4332-baab-4418cb7e72bc\") " pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:47.594958 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.594941 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmmhs\" (UniqueName: \"kubernetes.io/projected/7344905a-1586-4418-b3af-c26fee920038-kube-api-access-wmmhs\") pod \"iptables-alerter-nlq4q\" (UID: \"7344905a-1586-4418-b3af-c26fee920038\") " pod="openshift-network-operator/iptables-alerter-nlq4q" Sep 07 12:52:47.595020 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.595005 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72fj9\" (UniqueName: \"kubernetes.io/projected/f87025ad-1367-41bd-a774-3f27dc223933-kube-api-access-72fj9\") pod \"ovnkube-node-lpc6s\" (UID: \"f87025ad-1367-41bd-a774-3f27dc223933\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.654322 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:47.654296 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda94a700a9764e0ad82bdf73a6f15aedb.slice/crio-598a65bec1ebb28aa132b1773039e850b45ee43df0faaf7d8cfe3f999dd43313 WatchSource:0}: Error finding container 598a65bec1ebb28aa132b1773039e850b45ee43df0faaf7d8cfe3f999dd43313: Status 404 returned error can't find the container with id 598a65bec1ebb28aa132b1773039e850b45ee43df0faaf7d8cfe3f999dd43313 Sep 07 12:52:47.697517 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.697500 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vnn5z" Sep 07 12:52:47.706043 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:47.706020 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c872390_d8c4_46eb_9799_cbde30173723.slice/crio-0743b9b7b71e2275156ae7d54b9a391e00ef5f696dc316d1a3ba152d097c39cf WatchSource:0}: Error finding container 0743b9b7b71e2275156ae7d54b9a391e00ef5f696dc316d1a3ba152d097c39cf: Status 404 returned error can't find the container with id 0743b9b7b71e2275156ae7d54b9a391e00ef5f696dc316d1a3ba152d097c39cf Sep 07 12:52:47.722136 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.722114 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" Sep 07 12:52:47.729765 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:47.729747 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6d7f4fb_dd5a_4680_bfc1_b40129127fe7.slice/crio-702eff4da403aee1293f239022315dc550e0fe9bf114159bdd1377344d1c8a52 WatchSource:0}: Error finding container 702eff4da403aee1293f239022315dc550e0fe9bf114159bdd1377344d1c8a52: Status 404 returned error can't find the container with id 702eff4da403aee1293f239022315dc550e0fe9bf114159bdd1377344d1c8a52 Sep 07 12:52:47.743681 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.743663 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2lm8h" Sep 07 12:52:47.748741 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.748721 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nlq4q" Sep 07 12:52:47.751903 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:47.751885 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7296228_d10d_43e8_9c48_d6d72e58532a.slice/crio-204fd0ab19fb8dbc43458a3218b0e44bb40a57d65b38c7472ea70782e6ff4ea1 WatchSource:0}: Error finding container 204fd0ab19fb8dbc43458a3218b0e44bb40a57d65b38c7472ea70782e6ff4ea1: Status 404 returned error can't find the container with id 204fd0ab19fb8dbc43458a3218b0e44bb40a57d65b38c7472ea70782e6ff4ea1 Sep 07 12:52:47.753149 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.753122 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" Sep 07 12:52:47.757137 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:47.757116 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7344905a_1586_4418_b3af_c26fee920038.slice/crio-fe64c66b26e5f0381ccaf4cbfd7e27aa45cbdceae059b3778cb0a0a60b49a118 WatchSource:0}: Error finding container fe64c66b26e5f0381ccaf4cbfd7e27aa45cbdceae059b3778cb0a0a60b49a118: Status 404 returned error can't find the container with id fe64c66b26e5f0381ccaf4cbfd7e27aa45cbdceae059b3778cb0a0a60b49a118 Sep 07 12:52:47.759797 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.759778 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mmqsg" Sep 07 12:52:47.763255 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:47.763237 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb13984d5_8eb4_4fe8_ba4d_2d6a14b5b502.slice/crio-223cf931e1c832fd2dd01e0d458768116df7b48d957b831dce9d9caaf5665059 WatchSource:0}: Error finding container 223cf931e1c832fd2dd01e0d458768116df7b48d957b831dce9d9caaf5665059: Status 404 returned error can't find the container with id 223cf931e1c832fd2dd01e0d458768116df7b48d957b831dce9d9caaf5665059 Sep 07 12:52:47.763908 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.763896 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6dj5g" Sep 07 12:52:47.768019 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:47.767995 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8840b2d7_5f57_4887_9d76_1966edbec149.slice/crio-c41fd8af2863bb19eb2433ef4cfcb276339ca8686e63cf3985d290812f791059 WatchSource:0}: Error finding container c41fd8af2863bb19eb2433ef4cfcb276339ca8686e63cf3985d290812f791059: Status 404 returned error can't find the container with id c41fd8af2863bb19eb2433ef4cfcb276339ca8686e63cf3985d290812f791059 Sep 07 12:52:47.769572 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.769559 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vgpxq" Sep 07 12:52:47.772232 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:47.772215 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67ad982a_f807_4a65_b78d_8427d008e9cd.slice/crio-0984c6891b8878b24f55228f57ce7b50fed89a55fc42f91c9a95fc35b95c977d WatchSource:0}: Error finding container 0984c6891b8878b24f55228f57ce7b50fed89a55fc42f91c9a95fc35b95c977d: Status 404 returned error can't find the container with id 0984c6891b8878b24f55228f57ce7b50fed89a55fc42f91c9a95fc35b95c977d Sep 07 12:52:47.774141 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.774100 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:52:47.778598 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:47.778573 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a5430f6_3492_4a63_8126_d2080e0dcc3c.slice/crio-f562828697141eb759ea25925fb21539af994732be72d82c22951be2626910a4 WatchSource:0}: Error finding container f562828697141eb759ea25925fb21539af994732be72d82c22951be2626910a4: Status 404 returned error can't find the container with id f562828697141eb759ea25925fb21539af994732be72d82c22951be2626910a4 Sep 07 12:52:47.782724 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:52:47.782708 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf87025ad_1367_41bd_a774_3f27dc223933.slice/crio-fab97dd884b88aad9e8e13a9672ce0fb055b87a486ea761f3a26ceeb80f937cd WatchSource:0}: Error finding container fab97dd884b88aad9e8e13a9672ce0fb055b87a486ea761f3a26ceeb80f937cd: Status 404 returned error can't find the container with id fab97dd884b88aad9e8e13a9672ce0fb055b87a486ea761f3a26ceeb80f937cd Sep 07 12:52:47.907707 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.907626 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 12:52:47.987737 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:47.987707 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:47.987882 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.987859 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 12:52:47.987920 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:47.987912 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret podName:6e766773-bcc2-4cb6-8df6-3dc08e9d59cc nodeName:}" failed. No retries permitted until 2026-09-07 12:52:48.987899727 +0000 UTC m=+3.018649477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret") pod "global-pull-secret-syncer-hpq9f" (UID: "6e766773-bcc2-4cb6-8df6-3dc08e9d59cc") : object "kube-system"/"original-pull-secret" not registered Sep 07 12:52:48.088494 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.088462 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs\") pod \"network-metrics-daemon-pkv6v\" (UID: \"da204f77-d893-4332-baab-4418cb7e72bc\") " pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:48.088634 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:48.088615 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:52:48.088688 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:48.088682 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs podName:da204f77-d893-4332-baab-4418cb7e72bc nodeName:}" failed. No retries permitted until 2026-09-07 12:52:49.08866736 +0000 UTC m=+3.119417109 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs") pod "network-metrics-daemon-pkv6v" (UID: "da204f77-d893-4332-baab-4418cb7e72bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:52:48.189863 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.189773 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtvwt\" (UniqueName: \"kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt\") pod \"network-check-target-m9c7v\" (UID: \"7b8afbb2-9557-4821-8712-89cc16f472f4\") " pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:52:48.190077 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:48.190037 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 12:52:48.190077 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:48.190066 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 12:52:48.190223 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:48.190082 2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtvwt for pod openshift-network-diagnostics/network-check-target-m9c7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:52:48.190223 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:48.190166 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt podName:7b8afbb2-9557-4821-8712-89cc16f472f4 nodeName:}" failed. No retries permitted until 2026-09-07 12:52:49.19014553 +0000 UTC m=+3.220895286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtvwt" (UniqueName: "kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt") pod "network-check-target-m9c7v" (UID: "7b8afbb2-9557-4821-8712-89cc16f472f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:52:48.404284 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.404041 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 12:52:48.426005 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.425304 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 12:47:47 +0000 UTC" deadline="2028-02-10 11:55:27.379821243 +0000 UTC" Sep 07 12:52:48.426005 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.425360 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12503h2m38.954465843s" Sep 07 12:52:48.500005 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.499919 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6dj5g" event={"ID":"67ad982a-f807-4a65-b78d-8427d008e9cd","Type":"ContainerStarted","Data":"0984c6891b8878b24f55228f57ce7b50fed89a55fc42f91c9a95fc35b95c977d"} Sep 07 12:52:48.507636 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.507540 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mmqsg" event={"ID":"8840b2d7-5f57-4887-9d76-1966edbec149","Type":"ContainerStarted","Data":"c41fd8af2863bb19eb2433ef4cfcb276339ca8686e63cf3985d290812f791059"} Sep 07 12:52:48.520771 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.520602 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2lm8h" event={"ID":"a7296228-d10d-43e8-9c48-d6d72e58532a","Type":"ContainerStarted","Data":"204fd0ab19fb8dbc43458a3218b0e44bb40a57d65b38c7472ea70782e6ff4ea1"} Sep 07 12:52:48.529184 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.529126 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" event={"ID":"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7","Type":"ContainerStarted","Data":"702eff4da403aee1293f239022315dc550e0fe9bf114159bdd1377344d1c8a52"} Sep 07 12:52:48.532125 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.532095 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vnn5z" event={"ID":"8c872390-d8c4-46eb-9799-cbde30173723","Type":"ContainerStarted","Data":"0743b9b7b71e2275156ae7d54b9a391e00ef5f696dc316d1a3ba152d097c39cf"} Sep 07 12:52:48.534194 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.534166 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal" event={"ID":"a94a700a9764e0ad82bdf73a6f15aedb","Type":"ContainerStarted","Data":"598a65bec1ebb28aa132b1773039e850b45ee43df0faaf7d8cfe3f999dd43313"} Sep 07 12:52:48.546803 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.546773 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" event={"ID":"f87025ad-1367-41bd-a774-3f27dc223933","Type":"ContainerStarted","Data":"fab97dd884b88aad9e8e13a9672ce0fb055b87a486ea761f3a26ceeb80f937cd"} Sep 07 12:52:48.562199 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.562170 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vgpxq" event={"ID":"6a5430f6-3492-4a63-8126-d2080e0dcc3c","Type":"ContainerStarted","Data":"f562828697141eb759ea25925fb21539af994732be72d82c22951be2626910a4"} Sep 07 12:52:48.580725 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.580694 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" event={"ID":"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502","Type":"ContainerStarted","Data":"223cf931e1c832fd2dd01e0d458768116df7b48d957b831dce9d9caaf5665059"} Sep 07 12:52:48.583938 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.583911 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nlq4q" event={"ID":"7344905a-1586-4418-b3af-c26fee920038","Type":"ContainerStarted","Data":"fe64c66b26e5f0381ccaf4cbfd7e27aa45cbdceae059b3778cb0a0a60b49a118"} Sep 07 12:52:48.996600 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:48.995904 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:48.996600 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:48.996124 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 12:52:48.996600 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:48.996188 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret podName:6e766773-bcc2-4cb6-8df6-3dc08e9d59cc nodeName:}" failed. No retries permitted until 2026-09-07 12:52:50.996167731 +0000 UTC m=+5.026917490 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret") pod "global-pull-secret-syncer-hpq9f" (UID: "6e766773-bcc2-4cb6-8df6-3dc08e9d59cc") : object "kube-system"/"original-pull-secret" not registered Sep 07 12:52:49.097830 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:49.097231 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs\") pod \"network-metrics-daemon-pkv6v\" (UID: \"da204f77-d893-4332-baab-4418cb7e72bc\") " pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:49.097830 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:49.097420 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:52:49.097830 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:49.097484 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs podName:da204f77-d893-4332-baab-4418cb7e72bc nodeName:}" failed. No retries permitted until 2026-09-07 12:52:51.09746474 +0000 UTC m=+5.128214497 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs") pod "network-metrics-daemon-pkv6v" (UID: "da204f77-d893-4332-baab-4418cb7e72bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:52:49.197989 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:49.197933 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtvwt\" (UniqueName: \"kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt\") pod \"network-check-target-m9c7v\" (UID: \"7b8afbb2-9557-4821-8712-89cc16f472f4\") " pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:52:49.198180 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:49.198137 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 12:52:49.198180 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:49.198157 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 12:52:49.198180 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:49.198170 2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtvwt for pod openshift-network-diagnostics/network-check-target-m9c7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:52:49.198341 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:49.198232 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt podName:7b8afbb2-9557-4821-8712-89cc16f472f4 nodeName:}" failed. No retries permitted until 2026-09-07 12:52:51.198214168 +0000 UTC m=+5.228963927 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtvwt" (UniqueName: "kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt") pod "network-check-target-m9c7v" (UID: "7b8afbb2-9557-4821-8712-89cc16f472f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:52:49.427077 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:49.426886 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 12:47:47 +0000 UTC" deadline="2028-05-01 19:07:40.687850613 +0000 UTC" Sep 07 12:52:49.427077 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:49.426931 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14454h14m51.260925248s" Sep 07 12:52:49.476461 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:49.476430 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:49.476461 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:49.476453 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:52:49.476752 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:49.476594 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:52:49.476752 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:49.476689 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:52:49.476884 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:49.476764 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:49.476884 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:49.476866 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:52:49.995717 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:49.995685 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 12:52:51.014269 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:51.014229 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:51.014921 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:51.014400 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 12:52:51.014921 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:51.014469 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret podName:6e766773-bcc2-4cb6-8df6-3dc08e9d59cc nodeName:}" failed. No retries permitted until 2026-09-07 12:52:55.014448582 +0000 UTC m=+9.045198367 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret") pod "global-pull-secret-syncer-hpq9f" (UID: "6e766773-bcc2-4cb6-8df6-3dc08e9d59cc") : object "kube-system"/"original-pull-secret" not registered Sep 07 12:52:51.115581 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:51.115531 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs\") pod \"network-metrics-daemon-pkv6v\" (UID: \"da204f77-d893-4332-baab-4418cb7e72bc\") " pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:51.116016 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:51.115965 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:52:51.116137 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:51.116090 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs podName:da204f77-d893-4332-baab-4418cb7e72bc nodeName:}" failed. No retries permitted until 2026-09-07 12:52:55.116066306 +0000 UTC m=+9.146816071 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs") pod "network-metrics-daemon-pkv6v" (UID: "da204f77-d893-4332-baab-4418cb7e72bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:52:51.216760 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:51.216718 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtvwt\" (UniqueName: \"kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt\") pod \"network-check-target-m9c7v\" (UID: \"7b8afbb2-9557-4821-8712-89cc16f472f4\") " pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:52:51.216971 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:51.216950 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 12:52:51.217038 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:51.216983 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 12:52:51.217038 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:51.216998 2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtvwt for pod openshift-network-diagnostics/network-check-target-m9c7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:52:51.217211 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:51.217095 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt podName:7b8afbb2-9557-4821-8712-89cc16f472f4 nodeName:}" failed. No retries permitted until 2026-09-07 12:52:55.217075996 +0000 UTC m=+9.247825771 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtvwt" (UniqueName: "kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt") pod "network-check-target-m9c7v" (UID: "7b8afbb2-9557-4821-8712-89cc16f472f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:52:51.476439 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:51.476353 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:52:51.476625 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:51.476496 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:52:51.476831 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:51.476808 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:51.476964 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:51.476941 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:52:51.477068 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:51.477027 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:51.477145 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:51.477116 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:52:53.476483 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:53.476429 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:53.477144 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:53.476568 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:52:53.477144 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:53.476996 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:53.477144 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:53.477122 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:52:53.477336 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:53.477195 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:52:53.477336 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:53.477269 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:52:55.045684 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:55.045644 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:55.046183 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:55.045818 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 12:52:55.046183 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:55.045883 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret podName:6e766773-bcc2-4cb6-8df6-3dc08e9d59cc nodeName:}" failed. No retries permitted until 2026-09-07 12:53:03.045866056 +0000 UTC m=+17.076615810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret") pod "global-pull-secret-syncer-hpq9f" (UID: "6e766773-bcc2-4cb6-8df6-3dc08e9d59cc") : object "kube-system"/"original-pull-secret" not registered Sep 07 12:52:55.147216 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:55.146387 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs\") pod \"network-metrics-daemon-pkv6v\" (UID: \"da204f77-d893-4332-baab-4418cb7e72bc\") " pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:55.147216 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:55.146628 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:52:55.147216 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:55.146750 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs podName:da204f77-d893-4332-baab-4418cb7e72bc nodeName:}" failed. No retries permitted until 2026-09-07 12:53:03.146728079 +0000 UTC m=+17.177477828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs") pod "network-metrics-daemon-pkv6v" (UID: "da204f77-d893-4332-baab-4418cb7e72bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:52:55.247236 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:55.247194 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtvwt\" (UniqueName: \"kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt\") pod \"network-check-target-m9c7v\" (UID: \"7b8afbb2-9557-4821-8712-89cc16f472f4\") " pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:52:55.247418 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:55.247384 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 12:52:55.247418 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:55.247403 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 12:52:55.247418 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:55.247416 2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtvwt for pod openshift-network-diagnostics/network-check-target-m9c7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:52:55.247548 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:55.247474 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt podName:7b8afbb2-9557-4821-8712-89cc16f472f4 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:03.24745674 +0000 UTC m=+17.278206495 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtvwt" (UniqueName: "kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt") pod "network-check-target-m9c7v" (UID: "7b8afbb2-9557-4821-8712-89cc16f472f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:52:55.476697 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:55.475820 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:52:55.476697 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:55.475948 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:52:55.476697 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:55.476359 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:55.476697 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:55.476467 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:52:55.476697 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:55.476518 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:55.476697 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:55.476599 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:52:57.476695 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:57.476658 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:52:57.476695 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:57.476684 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:57.477200 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:57.476684 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:57.477200 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:57.476796 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:52:57.477200 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:57.476876 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:52:57.477200 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:57.476986 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:52:59.476124 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:59.476093 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:52:59.476564 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:59.476192 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:52:59.476564 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:59.476197 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:52:59.476564 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:52:59.476221 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:52:59.476564 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:59.476288 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:52:59.476564 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:52:59.476371 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:53:01.476314 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:01.476283 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:01.476744 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:01.476405 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:01.476744 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:01.476413 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:53:01.476744 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:01.476441 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:01.476744 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:01.476536 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:53:01.476744 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:01.476616 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:53:03.113040 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:03.113003 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:03.113579 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:03.113170 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 12:53:03.113579 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:03.113229 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret podName:6e766773-bcc2-4cb6-8df6-3dc08e9d59cc nodeName:}" failed. No retries permitted until 2026-09-07 12:53:19.113213626 +0000 UTC m=+33.143963380 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret") pod "global-pull-secret-syncer-hpq9f" (UID: "6e766773-bcc2-4cb6-8df6-3dc08e9d59cc") : object "kube-system"/"original-pull-secret" not registered Sep 07 12:53:03.214122 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:03.214093 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs\") pod \"network-metrics-daemon-pkv6v\" (UID: \"da204f77-d893-4332-baab-4418cb7e72bc\") " pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:03.214272 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:03.214242 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:53:03.214312 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:03.214300 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs podName:da204f77-d893-4332-baab-4418cb7e72bc nodeName:}" failed. No retries permitted until 2026-09-07 12:53:19.214285121 +0000 UTC m=+33.245034869 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs") pod "network-metrics-daemon-pkv6v" (UID: "da204f77-d893-4332-baab-4418cb7e72bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:53:03.315084 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:03.315051 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtvwt\" (UniqueName: \"kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt\") pod \"network-check-target-m9c7v\" (UID: \"7b8afbb2-9557-4821-8712-89cc16f472f4\") " pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:03.315246 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:03.315223 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 12:53:03.315246 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:03.315242 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 12:53:03.315311 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:03.315251 2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtvwt for pod openshift-network-diagnostics/network-check-target-m9c7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:53:03.315311 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:03.315300 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt podName:7b8afbb2-9557-4821-8712-89cc16f472f4 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:19.31528651 +0000 UTC m=+33.346036259 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtvwt" (UniqueName: "kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt") pod "network-check-target-m9c7v" (UID: "7b8afbb2-9557-4821-8712-89cc16f472f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:53:03.476833 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:03.476747 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:03.476833 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:03.476814 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:03.477051 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:03.476934 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:03.477051 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:03.476950 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:53:03.477051 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:03.477032 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:53:03.477154 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:03.477094 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:53:05.476021 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:05.475986 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:05.476475 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:05.476002 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:05.476475 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:05.476110 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:53:05.476475 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:05.476128 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:05.476475 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:05.476261 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:53:05.476475 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:05.476364 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:53:07.476407 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:07.476377 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:07.476407 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:07.476397 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:07.476774 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:07.476382 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:07.476774 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:07.476477 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:53:07.476774 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:07.476550 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:53:07.476774 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:07.476648 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:53:08.644240 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.643870 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-1.ec2.internal" event={"ID":"4351a43c17868aab7e62d03e90366ae5","Type":"ContainerStarted","Data":"e7e876e34d1d02fad68e8149116e5ec813481027fbd24942bf1aedaed88bdb75"} Sep 07 12:53:08.645567 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.645543 2576 generic.go:356] "Generic (PLEG): container finished" podID="67ad982a-f807-4a65-b78d-8427d008e9cd" containerID="c3bef7ec62f326e466defa02869ca5ac25340fe7691288609af0d8bafdd36e8a" exitCode=0 Sep 07 12:53:08.645666 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.645622 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6dj5g" event={"ID":"67ad982a-f807-4a65-b78d-8427d008e9cd","Type":"ContainerDied","Data":"c3bef7ec62f326e466defa02869ca5ac25340fe7691288609af0d8bafdd36e8a"} Sep 07 12:53:08.647176 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.647147 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mmqsg" event={"ID":"8840b2d7-5f57-4887-9d76-1966edbec149","Type":"ContainerStarted","Data":"336dde3f8ec6046ccf678ebb5a9118f8769eecb440abe80592d7e4a44e4e737a"} Sep 07 12:53:08.648703 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.648677 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2lm8h" event={"ID":"a7296228-d10d-43e8-9c48-d6d72e58532a","Type":"ContainerStarted","Data":"4085ec8f3d2eedaa050806ca6aa803d6f34152dbe1c2057aa96496f3ce070d94"} Sep 07 12:53:08.650169 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.650148 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" event={"ID":"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7","Type":"ContainerStarted","Data":"392bd19e0167dbc5c3d1beb0528c1b79ae8c6562c65b07ef5b83b3c613543daa"} Sep 07 12:53:08.651572 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.651543 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vnn5z" event={"ID":"8c872390-d8c4-46eb-9799-cbde30173723","Type":"ContainerStarted","Data":"ab7fa5c69af6a459ba39536a91afd3b4b6719fc24e137b9a000bfccf64a1c04c"} Sep 07 12:53:08.653254 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.653236 2576 generic.go:356] "Generic (PLEG): container finished" podID="a94a700a9764e0ad82bdf73a6f15aedb" containerID="ca564087e4993a124ee47241fcee9095bf7b79a945fb607c137f380547f98239" exitCode=0 Sep 07 12:53:08.653324 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.653303 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal" event={"ID":"a94a700a9764e0ad82bdf73a6f15aedb","Type":"ContainerDied","Data":"ca564087e4993a124ee47241fcee9095bf7b79a945fb607c137f380547f98239"} Sep 07 12:53:08.658268 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.658246 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" event={"ID":"f87025ad-1367-41bd-a774-3f27dc223933","Type":"ContainerStarted","Data":"39ecb42303a756668078ea96b5c4bee165fa33e878fb4b5dc4da59ba8fe013d4"} Sep 07 12:53:08.658375 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.658306 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" event={"ID":"f87025ad-1367-41bd-a774-3f27dc223933","Type":"ContainerStarted","Data":"ead2139345d46421d74c82a8db9c8cf9fbf5ab79d55ff012a5ba2fa737006c55"} Sep 07 12:53:08.658375 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.658354 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" event={"ID":"f87025ad-1367-41bd-a774-3f27dc223933","Type":"ContainerStarted","Data":"d6b9e04c6ab0084f0aee47fd6a27db9a98ecc487b0eff03313eda038a2e9e766"} Sep 07 12:53:08.658482 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.658393 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" event={"ID":"f87025ad-1367-41bd-a774-3f27dc223933","Type":"ContainerStarted","Data":"bac39640d4e77e8b1994bcb8c11a42b29409018cee2709925ca957baaf5022c1"} Sep 07 12:53:08.658482 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.658440 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" event={"ID":"f87025ad-1367-41bd-a774-3f27dc223933","Type":"ContainerStarted","Data":"c73d23a38c3625e3b08988797d532883704f49f80b4a7a3e099729fab0a2054b"} Sep 07 12:53:08.658575 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.658484 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" event={"ID":"f87025ad-1367-41bd-a774-3f27dc223933","Type":"ContainerStarted","Data":"bafb8e0c90600dbe247f6e7482c22b1afbfc73f7564da429aa49403850925281"} Sep 07 12:53:08.658681 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.658646 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-1.ec2.internal" podStartSLOduration=21.658633782 podStartE2EDuration="21.658633782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:08.658551351 +0000 UTC m=+22.689301123" watchObservedRunningTime="2026-09-07 12:53:08.658633782 +0000 UTC m=+22.689383552" Sep 07 12:53:08.659787 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.659769 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vgpxq" event={"ID":"6a5430f6-3492-4a63-8126-d2080e0dcc3c","Type":"ContainerStarted","Data":"5d841e85efeea5ceb4116506b24fe5c47f6b3d86ac4d94f934f82f8134a5e980"} Sep 07 12:53:08.661148 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.661125 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" event={"ID":"b13984d5-8eb4-4fe8-ba4d-2d6a14b5b502","Type":"ContainerStarted","Data":"83dacfdbe842893f1874ef681920ba0660f32f981a32e736ae649a726f89d541"} Sep 07 12:53:08.691017 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.690964 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vnn5z" podStartSLOduration=1.706699644 podStartE2EDuration="21.690952047s" totalImagesPullingTime="19.984252403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:08.676793899 +0000 UTC m=+22.707543703" watchObservedRunningTime="2026-09-07 12:53:08.690952047 +0000 UTC m=+22.721701889" Sep 07 12:53:08.691127 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.691108 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2lm8h" podStartSLOduration=1.756077112 podStartE2EDuration="21.691103863s" totalImagesPullingTime="19.935026751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:08.690645785 +0000 UTC m=+22.721395555" watchObservedRunningTime="2026-09-07 12:53:08.691103863 +0000 UTC m=+22.721853633" Sep 07 12:53:08.703387 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.703357 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mmqsg" podStartSLOduration=1.784478955 podStartE2EDuration="21.703346858s" totalImagesPullingTime="19.918867903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:08.70306429 +0000 UTC m=+22.733814060" watchObservedRunningTime="2026-09-07 12:53:08.703346858 +0000 UTC m=+22.734096628" Sep 07 12:53:08.736565 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.736529 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-t8qbk" podStartSLOduration=1.79512237 podStartE2EDuration="21.736517493s" totalImagesPullingTime="19.941395123s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:08.736262829 +0000 UTC m=+22.767012600" watchObservedRunningTime="2026-09-07 12:53:08.736517493 +0000 UTC m=+22.767267264" Sep 07 12:53:08.752026 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:08.751836 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vgpxq" podStartSLOduration=1.7832317020000001 podStartE2EDuration="21.751823491s" totalImagesPullingTime="19.968591789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:08.751726246 +0000 UTC m=+22.782476019" watchObservedRunningTime="2026-09-07 12:53:08.751823491 +0000 UTC m=+22.782573260" Sep 07 12:53:09.476799 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:09.476766 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:09.476799 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:09.476790 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:09.477105 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:09.476900 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:53:09.477105 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:09.476955 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:09.477105 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:09.477046 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:53:09.477315 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:09.477110 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:53:09.667894 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:09.667855 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nlq4q" event={"ID":"7344905a-1586-4418-b3af-c26fee920038","Type":"ContainerStarted","Data":"a2d30dbc427cf28e96e3b5e41d0161d59f2b9a39474008fff734398615fa624b"} Sep 07 12:53:09.684397 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:09.684352 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nlq4q" podStartSLOduration=2.749429755 podStartE2EDuration="22.684337553s" totalImagesPullingTime="19.934907798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:09.683808086 +0000 UTC m=+23.714557858" watchObservedRunningTime="2026-09-07 12:53:09.684337553 +0000 UTC m=+23.715087364" Sep 07 12:53:10.674319 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:10.674146 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal" event={"ID":"a94a700a9764e0ad82bdf73a6f15aedb","Type":"ContainerStarted","Data":"cd49a678ab48f2eb5343b44495565f4b98f7c6470a5076ad97e2ab5c5e159089"} Sep 07 12:53:10.680988 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:10.680945 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" event={"ID":"f87025ad-1367-41bd-a774-3f27dc223933","Type":"ContainerStarted","Data":"7537a45504ccc8e57705462b982bf9ad541e0ee219747d03b13c9037fb46ee97"} Sep 07 12:53:10.885191 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:10.885169 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 07 12:53:11.447073 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:11.446944 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T12:53:10.885187015Z","UUID":"b3703c75-0157-49e8-a168-f79e28e8a79c","Handler":null,"Name":"","Endpoint":""} Sep 07 12:53:11.449315 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:11.449144 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 07 12:53:11.449315 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:11.449177 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 07 12:53:11.476303 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:11.476274 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:11.476439 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:11.476379 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:11.476439 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:11.476383 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:53:11.476556 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:11.476477 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:53:11.476556 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:11.476387 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:11.476645 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:11.476582 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:53:11.632886 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:11.632846 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vnn5z" Sep 07 12:53:11.633771 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:11.633741 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vnn5z" Sep 07 12:53:11.652071 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:11.652020 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-1.ec2.internal" podStartSLOduration=24.652005473 podStartE2EDuration="24.652005473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:10.692220247 +0000 UTC m=+24.722970038" watchObservedRunningTime="2026-09-07 12:53:11.652005473 +0000 UTC m=+25.682755244" Sep 07 12:53:11.684761 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:11.684729 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" event={"ID":"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7","Type":"ContainerStarted","Data":"9bb70a11c726f85c045efc63b3bccb4183ea32e404e791e56ac79b137fe25077"} Sep 07 12:53:11.685167 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:11.684876 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vnn5z" Sep 07 12:53:11.685457 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:11.685414 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vnn5z" Sep 07 12:53:13.476638 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:13.476523 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:13.476638 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:13.476621 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:13.477246 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:13.476630 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:53:13.477246 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:13.476715 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:53:13.477246 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:13.476776 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:13.477246 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:13.476851 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:53:13.690814 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:13.690592 2576 generic.go:356] "Generic (PLEG): container finished" podID="67ad982a-f807-4a65-b78d-8427d008e9cd" containerID="6405610eb6da180bf62a9fbf7f763f14ef410381d2346300875c6fbb57199d38" exitCode=0 Sep 07 12:53:13.690814 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:13.690643 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6dj5g" event={"ID":"67ad982a-f807-4a65-b78d-8427d008e9cd","Type":"ContainerDied","Data":"6405610eb6da180bf62a9fbf7f763f14ef410381d2346300875c6fbb57199d38"} Sep 07 12:53:13.693118 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:13.693079 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" event={"ID":"d6d7f4fb-dd5a-4680-bfc1-b40129127fe7","Type":"ContainerStarted","Data":"c2b18021cb4a6fb026f88c0325c1c0661884439d0c7b48a2dea57495f3c0bd61"} Sep 07 12:53:13.696893 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:13.696866 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" event={"ID":"f87025ad-1367-41bd-a774-3f27dc223933","Type":"ContainerStarted","Data":"c6f7344da0d9b956f4c1b7ed1010b18d6571f25f2ed57dcb6facf84e434c517d"} Sep 07 12:53:13.697198 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:13.697175 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:53:13.712461 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:13.712441 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:53:13.803528 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:13.803474 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9vlkv" podStartSLOduration=1.837206384 podStartE2EDuration="26.80345975s" totalImagesPullingTime="24.966253366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:13.76116771 +0000 UTC m=+27.791917481" watchObservedRunningTime="2026-09-07 12:53:13.80345975 +0000 UTC m=+27.834209528" Sep 07 12:53:14.702004 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:14.701907 2576 generic.go:356] "Generic (PLEG): container finished" podID="67ad982a-f807-4a65-b78d-8427d008e9cd" containerID="fd520607a980872c1c65bd8bd5277259f1ab9248f26fb4cf76a1cfd8460bbd41" exitCode=0 Sep 07 12:53:14.702531 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:14.702014 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6dj5g" event={"ID":"67ad982a-f807-4a65-b78d-8427d008e9cd","Type":"ContainerDied","Data":"fd520607a980872c1c65bd8bd5277259f1ab9248f26fb4cf76a1cfd8460bbd41"} Sep 07 12:53:14.702789 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:14.702774 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:53:14.702863 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:14.702844 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:53:14.719478 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:14.718211 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:53:14.732964 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:14.732918 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" podStartSLOduration=7.758527437 podStartE2EDuration="27.732902675s" totalImagesPullingTime="19.974375238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:13.810077865 +0000 UTC m=+27.840827647" watchObservedRunningTime="2026-09-07 12:53:14.732902675 +0000 UTC m=+28.763652566" Sep 07 12:53:15.394752 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:15.394727 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m9c7v"] Sep 07 12:53:15.394892 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:15.394837 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:15.394962 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:15.394917 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:53:15.395436 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:15.395412 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hpq9f"] Sep 07 12:53:15.395620 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:15.395508 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:15.395683 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:15.395654 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:53:15.396029 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:15.396002 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pkv6v"] Sep 07 12:53:15.396142 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:15.396098 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:15.396198 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:15.396169 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:53:15.706381 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:15.706297 2576 generic.go:356] "Generic (PLEG): container finished" podID="67ad982a-f807-4a65-b78d-8427d008e9cd" containerID="04cf9c597e61bbf2703715413c04afff9ad94f96e9e999b161b9833d11424a85" exitCode=0 Sep 07 12:53:15.706938 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:15.706387 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6dj5g" event={"ID":"67ad982a-f807-4a65-b78d-8427d008e9cd","Type":"ContainerDied","Data":"04cf9c597e61bbf2703715413c04afff9ad94f96e9e999b161b9833d11424a85"} Sep 07 12:53:16.477566 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:16.477539 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:16.477738 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:16.477646 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:53:17.475857 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:17.475822 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:17.476287 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:17.475960 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:53:17.476287 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:17.476000 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:17.476287 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:17.476101 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:53:18.476699 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:18.476475 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:18.477215 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:18.476808 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:53:19.128089 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:19.128049 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:19.128273 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:19.128181 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 12:53:19.128273 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:19.128242 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret podName:6e766773-bcc2-4cb6-8df6-3dc08e9d59cc nodeName:}" failed. No retries permitted until 2026-09-07 12:53:51.12822683 +0000 UTC m=+65.158976579 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret") pod "global-pull-secret-syncer-hpq9f" (UID: "6e766773-bcc2-4cb6-8df6-3dc08e9d59cc") : object "kube-system"/"original-pull-secret" not registered Sep 07 12:53:19.228618 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:19.228582 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs\") pod \"network-metrics-daemon-pkv6v\" (UID: \"da204f77-d893-4332-baab-4418cb7e72bc\") " pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:19.228787 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:19.228746 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:53:19.228840 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:19.228820 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs podName:da204f77-d893-4332-baab-4418cb7e72bc nodeName:}" failed. No retries permitted until 2026-09-07 12:53:51.228801199 +0000 UTC m=+65.259550967 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs") pod "network-metrics-daemon-pkv6v" (UID: "da204f77-d893-4332-baab-4418cb7e72bc") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 12:53:19.329147 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:19.329107 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtvwt\" (UniqueName: \"kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt\") pod \"network-check-target-m9c7v\" (UID: \"7b8afbb2-9557-4821-8712-89cc16f472f4\") " pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:19.329317 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:19.329227 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 12:53:19.329317 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:19.329248 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 12:53:19.329317 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:19.329269 2576 projected.go:196] Error preparing data for projected volume kube-api-access-wtvwt for pod openshift-network-diagnostics/network-check-target-m9c7v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:53:19.329455 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:19.329326 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt podName:7b8afbb2-9557-4821-8712-89cc16f472f4 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:51.329312551 +0000 UTC m=+65.360062314 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wtvwt" (UniqueName: "kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt") pod "network-check-target-m9c7v" (UID: "7b8afbb2-9557-4821-8712-89cc16f472f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 12:53:19.475854 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:19.475774 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:19.476031 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:19.475887 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:53:19.476031 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:19.475990 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:19.476150 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:19.476125 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:53:20.479763 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:20.479693 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:20.480107 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:20.479798 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hpq9f" podUID="6e766773-bcc2-4cb6-8df6-3dc08e9d59cc" Sep 07 12:53:20.721496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:20.721462 2576 generic.go:356] "Generic (PLEG): container finished" podID="67ad982a-f807-4a65-b78d-8427d008e9cd" containerID="c31a31b8a84729f5a72c80c2cf740c2cbd71e7e25e12f15c978e3fe43ddfb3fb" exitCode=0 Sep 07 12:53:20.721669 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:20.721535 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6dj5g" event={"ID":"67ad982a-f807-4a65-b78d-8427d008e9cd","Type":"ContainerDied","Data":"c31a31b8a84729f5a72c80c2cf740c2cbd71e7e25e12f15c978e3fe43ddfb3fb"} Sep 07 12:53:21.475810 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.475782 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:21.476054 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.475819 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:21.476054 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:21.475920 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv6v" podUID="da204f77-d893-4332-baab-4418cb7e72bc" Sep 07 12:53:21.476181 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:21.476065 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m9c7v" podUID="7b8afbb2-9557-4821-8712-89cc16f472f4" Sep 07 12:53:21.603887 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.603857 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-1.ec2.internal" event="NodeReady" Sep 07 12:53:21.604248 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.603960 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 07 12:53:21.654720 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.654692 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-6kghh"] Sep 07 12:53:21.676229 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.676197 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-5fzst"] Sep 07 12:53:21.676394 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.676349 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-6kghh" Sep 07 12:53:21.679238 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.679216 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-sfdbb\"" Sep 07 12:53:21.679371 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.679218 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 07 12:53:21.679371 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.679271 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 07 12:53:21.691948 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.691926 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c6bcbc855-x2htj"] Sep 07 12:53:21.692169 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.692076 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:21.700458 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.695897 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 07 12:53:21.700458 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.695951 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 07 12:53:21.700458 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.695901 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 07 12:53:21.700458 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.696101 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-29qws\"" Sep 07 12:53:21.700458 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.696826 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 07 12:53:21.703073 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.703052 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 07 12:53:21.711986 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.711955 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-6kghh"] Sep 07 12:53:21.712070 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.712007 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt"] Sep 07 12:53:21.712070 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.712050 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.714035 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.714012 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 07 12:53:21.714141 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.714042 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v9hr7\"" Sep 07 12:53:21.714141 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.714015 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 07 12:53:21.714462 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.714446 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 07 12:53:21.721790 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.721771 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 07 12:53:21.727396 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.727377 2576 generic.go:356] "Generic (PLEG): container finished" podID="67ad982a-f807-4a65-b78d-8427d008e9cd" containerID="7de2cad5bfccde0bef6e229af8d82452434d26285fad822836b2bf8a084c357c" exitCode=0 Sep 07 12:53:21.731989 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.731952 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6dj5g" event={"ID":"67ad982a-f807-4a65-b78d-8427d008e9cd","Type":"ContainerDied","Data":"7de2cad5bfccde0bef6e229af8d82452434d26285fad822836b2bf8a084c357c"} Sep 07 12:53:21.732080 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.732011 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-5fzst"] Sep 07 12:53:21.732080 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.732041 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn"] Sep 07 12:53:21.732190 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.732084 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" Sep 07 12:53:21.734475 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.734461 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-m9jh7\"" Sep 07 12:53:21.734714 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.734697 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 07 12:53:21.734868 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.734855 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 07 12:53:21.734986 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.734958 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 07 12:53:21.735088 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.735073 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 07 12:53:21.748624 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.748605 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4561d14-f779-4e6f-b63c-09f5d09bafed-config\") pod \"console-operator-85b454874b-5fzst\" (UID: \"d4561d14-f779-4e6f-b63c-09f5d09bafed\") " pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:21.748707 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.748663 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnmk6\" (UniqueName: \"kubernetes.io/projected/d4561d14-f779-4e6f-b63c-09f5d09bafed-kube-api-access-gnmk6\") pod \"console-operator-85b454874b-5fzst\" (UID: \"d4561d14-f779-4e6f-b63c-09f5d09bafed\") " pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:21.748707 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.748682 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4ft9\" (UniqueName: \"kubernetes.io/projected/dc212698-c23a-4c52-83b5-137b94537bda-kube-api-access-w4ft9\") pod \"volume-data-source-validator-695c895bd6-6kghh\" (UID: \"dc212698-c23a-4c52-83b5-137b94537bda\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-6kghh" Sep 07 12:53:21.748707 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.748700 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4561d14-f779-4e6f-b63c-09f5d09bafed-serving-cert\") pod \"console-operator-85b454874b-5fzst\" (UID: \"d4561d14-f779-4e6f-b63c-09f5d09bafed\") " pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:21.748825 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.748715 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4561d14-f779-4e6f-b63c-09f5d09bafed-trusted-ca\") pod \"console-operator-85b454874b-5fzst\" (UID: \"d4561d14-f779-4e6f-b63c-09f5d09bafed\") " pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:21.750488 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.750472 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6998fdb454-4p474"] Sep 07 12:53:21.750595 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.750583 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" Sep 07 12:53:21.753332 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.753311 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 07 12:53:21.754371 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.754158 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 07 12:53:21.754482 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.754464 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 07 12:53:21.754871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.754853 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 07 12:53:21.754995 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.754891 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7vmcw\"" Sep 07 12:53:21.782993 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.782946 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-rmcsw"] Sep 07 12:53:21.783199 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.783178 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.785338 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.785322 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mns4n\"" Sep 07 12:53:21.785680 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.785665 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 07 12:53:21.785950 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.785930 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 07 12:53:21.786053 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.785953 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 07 12:53:21.786053 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.785983 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 07 12:53:21.786053 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.785933 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 07 12:53:21.786435 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.786420 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 07 12:53:21.795403 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.795386 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx"] Sep 07 12:53:21.795519 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.795504 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-rmcsw" Sep 07 12:53:21.798939 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.798922 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-fwkdw\"" Sep 07 12:53:21.799053 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.798990 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 07 12:53:21.799053 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.799021 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 07 12:53:21.817336 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.817319 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-jdjj8"] Sep 07 12:53:21.817431 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.817419 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:21.820920 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.820867 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 07 12:53:21.822194 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.821280 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-t55z9\"" Sep 07 12:53:21.822194 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.821414 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 07 12:53:21.822194 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.821510 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 07 12:53:21.822194 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.821547 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 07 12:53:21.841497 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.841479 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-8nr54"] Sep 07 12:53:21.841616 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.841601 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:21.843281 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.843263 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 07 12:53:21.843636 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.843613 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 07 12:53:21.843636 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.843617 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-v6g2k\"" Sep 07 12:53:21.849511 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849494 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.849621 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849561 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15ddd367-8b23-4e84-b27f-187d89b0a7bc-installation-pull-secrets\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.849689 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849628 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4561d14-f779-4e6f-b63c-09f5d09bafed-config\") pod \"console-operator-85b454874b-5fzst\" (UID: \"d4561d14-f779-4e6f-b63c-09f5d09bafed\") " pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:21.849689 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849670 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-bound-sa-token\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.849857 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849697 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-stats-auth\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.849857 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849739 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7b88887b-d35e-4197-987a-7c91b196cba8-serving-cert\") pod \"service-ca-operator-859f94c4b-trlrt\" (UID: \"7b88887b-d35e-4197-987a-7c91b196cba8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" Sep 07 12:53:21.849857 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849762 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c73cfc77-cf60-4c83-9d2c-1f5a56779218-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9r4xn\" (UID: \"c73cfc77-cf60-4c83-9d2c-1f5a56779218\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" Sep 07 12:53:21.849857 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849789 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15ddd367-8b23-4e84-b27f-187d89b0a7bc-image-registry-private-configuration\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.849857 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849843 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htpl4\" (UniqueName: \"kubernetes.io/projected/7b88887b-d35e-4197-987a-7c91b196cba8-kube-api-access-htpl4\") pod \"service-ca-operator-859f94c4b-trlrt\" (UID: \"7b88887b-d35e-4197-987a-7c91b196cba8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" Sep 07 12:53:21.850159 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849860 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndw9l\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-kube-api-access-ndw9l\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.850159 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849880 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnmk6\" (UniqueName: \"kubernetes.io/projected/d4561d14-f779-4e6f-b63c-09f5d09bafed-kube-api-access-gnmk6\") pod \"console-operator-85b454874b-5fzst\" (UID: \"d4561d14-f779-4e6f-b63c-09f5d09bafed\") " pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:21.850159 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849902 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.850159 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849925 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15ddd367-8b23-4e84-b27f-187d89b0a7bc-trusted-ca\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.850159 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.849956 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97ccc\" (UniqueName: \"kubernetes.io/projected/c73cfc77-cf60-4c83-9d2c-1f5a56779218-kube-api-access-97ccc\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9r4xn\" (UID: \"c73cfc77-cf60-4c83-9d2c-1f5a56779218\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" Sep 07 12:53:21.850159 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.850011 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4ft9\" (UniqueName: \"kubernetes.io/projected/dc212698-c23a-4c52-83b5-137b94537bda-kube-api-access-w4ft9\") pod \"volume-data-source-validator-695c895bd6-6kghh\" (UID: \"dc212698-c23a-4c52-83b5-137b94537bda\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-6kghh" Sep 07 12:53:21.850159 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.850036 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4561d14-f779-4e6f-b63c-09f5d09bafed-serving-cert\") pod \"console-operator-85b454874b-5fzst\" (UID: \"d4561d14-f779-4e6f-b63c-09f5d09bafed\") " pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:21.850159 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.850069 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nswtr\" (UniqueName: \"kubernetes.io/projected/f54978ff-9e19-427c-bec3-602c1ed25a1e-kube-api-access-nswtr\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.850159 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.850094 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4561d14-f779-4e6f-b63c-09f5d09bafed-trusted-ca\") pod \"console-operator-85b454874b-5fzst\" (UID: \"d4561d14-f779-4e6f-b63c-09f5d09bafed\") " pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:21.850159 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.850116 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.850159 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.850163 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7b88887b-d35e-4197-987a-7c91b196cba8-config\") pod \"service-ca-operator-859f94c4b-trlrt\" (UID: \"7b88887b-d35e-4197-987a-7c91b196cba8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" Sep 07 12:53:21.850644 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.850187 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15ddd367-8b23-4e84-b27f-187d89b0a7bc-ca-trust-extracted\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.850644 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.850209 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-certificates\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.850644 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.850268 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-default-certificate\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.850644 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.850351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c73cfc77-cf60-4c83-9d2c-1f5a56779218-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9r4xn\" (UID: \"c73cfc77-cf60-4c83-9d2c-1f5a56779218\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" Sep 07 12:53:21.850644 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.850359 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4561d14-f779-4e6f-b63c-09f5d09bafed-config\") pod \"console-operator-85b454874b-5fzst\" (UID: \"d4561d14-f779-4e6f-b63c-09f5d09bafed\") " pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:21.851152 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.851134 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4561d14-f779-4e6f-b63c-09f5d09bafed-trusted-ca\") pod \"console-operator-85b454874b-5fzst\" (UID: \"d4561d14-f779-4e6f-b63c-09f5d09bafed\") " pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:21.854088 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.853910 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4561d14-f779-4e6f-b63c-09f5d09bafed-serving-cert\") pod \"console-operator-85b454874b-5fzst\" (UID: \"d4561d14-f779-4e6f-b63c-09f5d09bafed\") " pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:21.859854 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.859836 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg"] Sep 07 12:53:21.860111 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.859955 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:21.862896 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.862681 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 07 12:53:21.862896 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.862712 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 07 12:53:21.862896 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.862774 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 07 12:53:21.862896 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.862804 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 07 12:53:21.863160 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.863081 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-w6l7x\"" Sep 07 12:53:21.863647 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.863627 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4ft9\" (UniqueName: \"kubernetes.io/projected/dc212698-c23a-4c52-83b5-137b94537bda-kube-api-access-w4ft9\") pod \"volume-data-source-validator-695c895bd6-6kghh\" (UID: \"dc212698-c23a-4c52-83b5-137b94537bda\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-6kghh" Sep 07 12:53:21.863729 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.863706 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnmk6\" (UniqueName: \"kubernetes.io/projected/d4561d14-f779-4e6f-b63c-09f5d09bafed-kube-api-access-gnmk6\") pod \"console-operator-85b454874b-5fzst\" (UID: \"d4561d14-f779-4e6f-b63c-09f5d09bafed\") " pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:21.868304 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.868284 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 07 12:53:21.876281 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.876257 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt"] Sep 07 12:53:21.876281 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.876284 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c6bcbc855-x2htj"] Sep 07 12:53:21.876394 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.876298 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-jdjj8"] Sep 07 12:53:21.876394 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.876313 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn"] Sep 07 12:53:21.876394 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.876325 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg"] Sep 07 12:53:21.876394 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.876341 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-rmcsw"] Sep 07 12:53:21.876394 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.876343 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:21.876394 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.876353 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-8nr54"] Sep 07 12:53:21.876394 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.876387 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bq8pr"] Sep 07 12:53:21.878336 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.878317 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 07 12:53:21.878435 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.878368 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-txsb2\"" Sep 07 12:53:21.878435 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.878394 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 07 12:53:21.878653 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.878639 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 07 12:53:21.894232 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.894216 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5zsmj"] Sep 07 12:53:21.894310 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.894299 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:21.898106 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.897268 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8c6z8\"" Sep 07 12:53:21.898106 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.897716 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 07 12:53:21.898106 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.897725 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 07 12:53:21.916788 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.916765 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:21.922315 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.922300 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6998fdb454-4p474"] Sep 07 12:53:21.922372 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.922326 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5zsmj"] Sep 07 12:53:21.922372 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.922340 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bq8pr"] Sep 07 12:53:21.922372 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.922357 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx"] Sep 07 12:53:21.922461 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.922390 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5zsmj" Sep 07 12:53:21.925822 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.925806 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 07 12:53:21.925961 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.925946 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 07 12:53:21.926093 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.926044 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 07 12:53:21.926158 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.926064 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-j64n2\"" Sep 07 12:53:21.950685 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.950662 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15ddd367-8b23-4e84-b27f-187d89b0a7bc-ca-trust-extracted\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.950787 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.950701 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-default-certificate\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.950787 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.950727 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c18ca5b5-32d3-4953-a134-397fce347a66-snapshots\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:21.950787 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.950744 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c73cfc77-cf60-4c83-9d2c-1f5a56779218-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9r4xn\" (UID: \"c73cfc77-cf60-4c83-9d2c-1f5a56779218\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" Sep 07 12:53:21.950787 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.950769 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.951031 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.950823 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15ddd367-8b23-4e84-b27f-187d89b0a7bc-installation-pull-secrets\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.951031 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:21.950919 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle podName:f54978ff-9e19-427c-bec3-602c1ed25a1e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:22.450901098 +0000 UTC m=+36.481650850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle") pod "router-default-6998fdb454-4p474" (UID: "f54978ff-9e19-427c-bec3-602c1ed25a1e") : configmap references non-existent config key: service-ca.crt Sep 07 12:53:21.951031 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.950952 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-config-volume\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:21.951202 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951075 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:21.951202 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951114 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-szxpg\" (UID: \"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:21.951202 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951144 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15ddd367-8b23-4e84-b27f-187d89b0a7bc-ca-trust-extracted\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.951202 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951173 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7b88887b-d35e-4197-987a-7c91b196cba8-serving-cert\") pod \"service-ca-operator-859f94c4b-trlrt\" (UID: \"7b88887b-d35e-4197-987a-7c91b196cba8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" Sep 07 12:53:21.951202 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951202 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15ddd367-8b23-4e84-b27f-187d89b0a7bc-image-registry-private-configuration\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.951428 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951242 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c18ca5b5-32d3-4953-a134-397fce347a66-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:21.951428 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951269 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c18ca5b5-32d3-4953-a134-397fce347a66-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:21.951428 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951292 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx9w2\" (UniqueName: \"kubernetes.io/projected/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-kube-api-access-tx9w2\") pod \"cluster-samples-operator-7cdcfcf76c-szxpg\" (UID: \"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:21.951428 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951321 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndw9l\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-kube-api-access-ndw9l\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.951428 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951367 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97ccc\" (UniqueName: \"kubernetes.io/projected/c73cfc77-cf60-4c83-9d2c-1f5a56779218-kube-api-access-97ccc\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9r4xn\" (UID: \"c73cfc77-cf60-4c83-9d2c-1f5a56779218\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" Sep 07 12:53:21.951428 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951408 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:21.951725 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951434 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c18ca5b5-32d3-4953-a134-397fce347a66-tmp\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:21.951725 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951482 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-tmp-dir\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:21.951725 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951506 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w6px\" (UniqueName: \"kubernetes.io/projected/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-kube-api-access-6w6px\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:21.951725 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951532 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbl9q\" (UniqueName: \"kubernetes.io/projected/703a67a7-558f-450b-9e14-d9c2d8a2109b-kube-api-access-sbl9q\") pod \"network-check-source-685456655f-rmcsw\" (UID: \"703a67a7-558f-450b-9e14-d9c2d8a2109b\") " pod="openshift-network-diagnostics/network-check-source-685456655f-rmcsw" Sep 07 12:53:21.951725 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951584 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7b88887b-d35e-4197-987a-7c91b196cba8-config\") pod \"service-ca-operator-859f94c4b-trlrt\" (UID: \"7b88887b-d35e-4197-987a-7c91b196cba8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" Sep 07 12:53:21.951725 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951611 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-certificates\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.951725 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951657 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d5881837-ea29-48f0-865e-40542a004cf5-nginx-conf\") pod \"networking-console-plugin-79977f9844-jdjj8\" (UID: \"d5881837-ea29-48f0-865e-40542a004cf5\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:21.951725 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951685 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-bound-sa-token\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.951725 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951712 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:21.952165 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.951741 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-stats-auth\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.952361 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.952297 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7b88887b-d35e-4197-987a-7c91b196cba8-config\") pod \"service-ca-operator-859f94c4b-trlrt\" (UID: \"7b88887b-d35e-4197-987a-7c91b196cba8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" Sep 07 12:53:21.952361 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.952351 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c73cfc77-cf60-4c83-9d2c-1f5a56779218-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9r4xn\" (UID: \"c73cfc77-cf60-4c83-9d2c-1f5a56779218\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.952388 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-584bj\" (UniqueName: \"kubernetes.io/projected/c18ca5b5-32d3-4953-a134-397fce347a66-kube-api-access-584bj\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.952490 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-certificates\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.952514 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9jz8\" (UniqueName: \"kubernetes.io/projected/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-kube-api-access-d9jz8\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.952593 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htpl4\" (UniqueName: \"kubernetes.io/projected/7b88887b-d35e-4197-987a-7c91b196cba8-kube-api-access-htpl4\") pod \"service-ca-operator-859f94c4b-trlrt\" (UID: \"7b88887b-d35e-4197-987a-7c91b196cba8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.952618 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15ddd367-8b23-4e84-b27f-187d89b0a7bc-trusted-ca\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.952648 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nswtr\" (UniqueName: \"kubernetes.io/projected/f54978ff-9e19-427c-bec3-602c1ed25a1e-kube-api-access-nswtr\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.952673 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.952697 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jdjj8\" (UID: \"d5881837-ea29-48f0-865e-40542a004cf5\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.952725 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.952751 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c18ca5b5-32d3-4953-a134-397fce347a66-serving-cert\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.952849 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c73cfc77-cf60-4c83-9d2c-1f5a56779218-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9r4xn\" (UID: \"c73cfc77-cf60-4c83-9d2c-1f5a56779218\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:21.952901 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:21.952992 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs podName:f54978ff-9e19-427c-bec3-602c1ed25a1e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:22.452947989 +0000 UTC m=+36.483697756 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs") pod "router-default-6998fdb454-4p474" (UID: "f54978ff-9e19-427c-bec3-602c1ed25a1e") : secret "router-metrics-certs-default" not found Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:21.953062 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:21.953074 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c6bcbc855-x2htj: secret "image-registry-tls" not found Sep 07 12:53:21.953313 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:21.953108 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls podName:15ddd367-8b23-4e84-b27f-187d89b0a7bc nodeName:}" failed. No retries permitted until 2026-09-07 12:53:22.453096139 +0000 UTC m=+36.483845920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls") pod "image-registry-c6bcbc855-x2htj" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc") : secret "image-registry-tls" not found Sep 07 12:53:21.954119 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.953735 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15ddd367-8b23-4e84-b27f-187d89b0a7bc-trusted-ca\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.954180 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.954137 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7b88887b-d35e-4197-987a-7c91b196cba8-serving-cert\") pod \"service-ca-operator-859f94c4b-trlrt\" (UID: \"7b88887b-d35e-4197-987a-7c91b196cba8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" Sep 07 12:53:21.954180 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.954162 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15ddd367-8b23-4e84-b27f-187d89b0a7bc-installation-pull-secrets\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.954269 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.954231 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-default-certificate\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.954386 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.954367 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c73cfc77-cf60-4c83-9d2c-1f5a56779218-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9r4xn\" (UID: \"c73cfc77-cf60-4c83-9d2c-1f5a56779218\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" Sep 07 12:53:21.954464 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.954447 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15ddd367-8b23-4e84-b27f-187d89b0a7bc-image-registry-private-configuration\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.955221 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.955203 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-stats-auth\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.964856 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.964837 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndw9l\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-kube-api-access-ndw9l\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.968105 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.968087 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nswtr\" (UniqueName: \"kubernetes.io/projected/f54978ff-9e19-427c-bec3-602c1ed25a1e-kube-api-access-nswtr\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:21.970183 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.970157 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97ccc\" (UniqueName: \"kubernetes.io/projected/c73cfc77-cf60-4c83-9d2c-1f5a56779218-kube-api-access-97ccc\") pod \"kube-storage-version-migrator-operator-8d7dd4675-9r4xn\" (UID: \"c73cfc77-cf60-4c83-9d2c-1f5a56779218\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" Sep 07 12:53:21.970297 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.970165 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-bound-sa-token\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:21.970464 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.970448 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htpl4\" (UniqueName: \"kubernetes.io/projected/7b88887b-d35e-4197-987a-7c91b196cba8-kube-api-access-htpl4\") pod \"service-ca-operator-859f94c4b-trlrt\" (UID: \"7b88887b-d35e-4197-987a-7c91b196cba8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" Sep 07 12:53:21.987623 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:21.987605 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-6kghh" Sep 07 12:53:22.003922 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.003905 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:22.015286 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.015261 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:22.040823 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.040796 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" Sep 07 12:53:22.054094 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054064 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd7a4a12-f4be-4941-89d8-7e68d809263b-cert\") pod \"ingress-canary-5zsmj\" (UID: \"cd7a4a12-f4be-4941-89d8-7e68d809263b\") " pod="openshift-ingress-canary/ingress-canary-5zsmj" Sep 07 12:53:22.054204 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054104 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d5881837-ea29-48f0-865e-40542a004cf5-nginx-conf\") pod \"networking-console-plugin-79977f9844-jdjj8\" (UID: \"d5881837-ea29-48f0-865e-40542a004cf5\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:22.054204 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054123 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:22.054204 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054153 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-584bj\" (UniqueName: \"kubernetes.io/projected/c18ca5b5-32d3-4953-a134-397fce347a66-kube-api-access-584bj\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:22.054204 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054181 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9jz8\" (UniqueName: \"kubernetes.io/projected/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-kube-api-access-d9jz8\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:22.054330 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054206 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z527l\" (UniqueName: \"kubernetes.io/projected/cd7a4a12-f4be-4941-89d8-7e68d809263b-kube-api-access-z527l\") pod \"ingress-canary-5zsmj\" (UID: \"cd7a4a12-f4be-4941-89d8-7e68d809263b\") " pod="openshift-ingress-canary/ingress-canary-5zsmj" Sep 07 12:53:22.054330 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.054232 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 07 12:53:22.054330 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054258 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jdjj8\" (UID: \"d5881837-ea29-48f0-865e-40542a004cf5\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:22.054330 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.054300 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls podName:5afdf94f-7a4f-483b-9f83-cab863ef2e7b nodeName:}" failed. No retries permitted until 2026-09-07 12:53:22.554284873 +0000 UTC m=+36.585034622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-n6cnx" (UID: "5afdf94f-7a4f-483b-9f83-cab863ef2e7b") : secret "cluster-monitoring-operator-tls" not found Sep 07 12:53:22.054330 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054317 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c18ca5b5-32d3-4953-a134-397fce347a66-serving-cert\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:22.054483 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.054355 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 07 12:53:22.054483 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054364 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c18ca5b5-32d3-4953-a134-397fce347a66-snapshots\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:22.054483 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054404 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-config-volume\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:22.054483 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054418 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:22.054483 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054436 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-szxpg\" (UID: \"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:22.054483 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.054463 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert podName:d5881837-ea29-48f0-865e-40542a004cf5 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:22.554445925 +0000 UTC m=+36.585195693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-jdjj8" (UID: "d5881837-ea29-48f0-865e-40542a004cf5") : secret "networking-console-plugin-cert" not found Sep 07 12:53:22.054483 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.054482 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 07 12:53:22.054682 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054504 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c18ca5b5-32d3-4953-a134-397fce347a66-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:22.054682 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.054519 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls podName:ac7c410e-42b2-4bcd-9099-9b8d7244cc5e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:22.554512509 +0000 UTC m=+36.585262258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-szxpg" (UID: "ac7c410e-42b2-4bcd-9099-9b8d7244cc5e") : secret "samples-operator-tls" not found Sep 07 12:53:22.054682 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054535 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c18ca5b5-32d3-4953-a134-397fce347a66-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:22.054682 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054551 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx9w2\" (UniqueName: \"kubernetes.io/projected/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-kube-api-access-tx9w2\") pod \"cluster-samples-operator-7cdcfcf76c-szxpg\" (UID: \"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:22.054682 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054569 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:22.054682 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054586 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c18ca5b5-32d3-4953-a134-397fce347a66-tmp\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:22.054682 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054602 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-tmp-dir\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:22.054682 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054617 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w6px\" (UniqueName: \"kubernetes.io/projected/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-kube-api-access-6w6px\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:22.054682 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054632 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbl9q\" (UniqueName: \"kubernetes.io/projected/703a67a7-558f-450b-9e14-d9c2d8a2109b-kube-api-access-sbl9q\") pod \"network-check-source-685456655f-rmcsw\" (UID: \"703a67a7-558f-450b-9e14-d9c2d8a2109b\") " pod="openshift-network-diagnostics/network-check-source-685456655f-rmcsw" Sep 07 12:53:22.055023 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.054849 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d5881837-ea29-48f0-865e-40542a004cf5-nginx-conf\") pod \"networking-console-plugin-79977f9844-jdjj8\" (UID: \"d5881837-ea29-48f0-865e-40542a004cf5\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:22.055456 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.055430 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-tmp-dir\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:22.055605 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.055493 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c18ca5b5-32d3-4953-a134-397fce347a66-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:22.055605 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.055561 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c18ca5b5-32d3-4953-a134-397fce347a66-snapshots\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:22.055605 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.055590 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-config-volume\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:22.055748 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.055610 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c18ca5b5-32d3-4953-a134-397fce347a66-tmp\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:22.055748 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.055682 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 12:53:22.055748 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.055725 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls podName:9fd689c9-28b9-4a27-a6a5-bf0133374ac8 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:22.555710815 +0000 UTC m=+36.586460569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls") pod "dns-default-bq8pr" (UID: "9fd689c9-28b9-4a27-a6a5-bf0133374ac8") : secret "dns-default-metrics-tls" not found Sep 07 12:53:22.056506 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.056261 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:22.058330 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.058307 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c18ca5b5-32d3-4953-a134-397fce347a66-serving-cert\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:22.062148 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.060704 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" Sep 07 12:53:22.064234 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.064215 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbl9q\" (UniqueName: \"kubernetes.io/projected/703a67a7-558f-450b-9e14-d9c2d8a2109b-kube-api-access-sbl9q\") pod \"network-check-source-685456655f-rmcsw\" (UID: \"703a67a7-558f-450b-9e14-d9c2d8a2109b\") " pod="openshift-network-diagnostics/network-check-source-685456655f-rmcsw" Sep 07 12:53:22.064381 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.064364 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx9w2\" (UniqueName: \"kubernetes.io/projected/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-kube-api-access-tx9w2\") pod \"cluster-samples-operator-7cdcfcf76c-szxpg\" (UID: \"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:22.066285 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.066265 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c18ca5b5-32d3-4953-a134-397fce347a66-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:22.071226 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.071172 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-584bj\" (UniqueName: \"kubernetes.io/projected/c18ca5b5-32d3-4953-a134-397fce347a66-kube-api-access-584bj\") pod \"insights-operator-76bcd6ccf5-8nr54\" (UID: \"c18ca5b5-32d3-4953-a134-397fce347a66\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:22.072371 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.072348 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w6px\" (UniqueName: \"kubernetes.io/projected/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-kube-api-access-6w6px\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:22.077420 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.077303 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9jz8\" (UniqueName: \"kubernetes.io/projected/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-kube-api-access-d9jz8\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:22.115750 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.115684 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-rmcsw" Sep 07 12:53:22.156013 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.155984 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd7a4a12-f4be-4941-89d8-7e68d809263b-cert\") pod \"ingress-canary-5zsmj\" (UID: \"cd7a4a12-f4be-4941-89d8-7e68d809263b\") " pod="openshift-ingress-canary/ingress-canary-5zsmj" Sep 07 12:53:22.156160 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.156051 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z527l\" (UniqueName: \"kubernetes.io/projected/cd7a4a12-f4be-4941-89d8-7e68d809263b-kube-api-access-z527l\") pod \"ingress-canary-5zsmj\" (UID: \"cd7a4a12-f4be-4941-89d8-7e68d809263b\") " pod="openshift-ingress-canary/ingress-canary-5zsmj" Sep 07 12:53:22.159067 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.159045 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd7a4a12-f4be-4941-89d8-7e68d809263b-cert\") pod \"ingress-canary-5zsmj\" (UID: \"cd7a4a12-f4be-4941-89d8-7e68d809263b\") " pod="openshift-ingress-canary/ingress-canary-5zsmj" Sep 07 12:53:22.166626 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.166574 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z527l\" (UniqueName: \"kubernetes.io/projected/cd7a4a12-f4be-4941-89d8-7e68d809263b-kube-api-access-z527l\") pod \"ingress-canary-5zsmj\" (UID: \"cd7a4a12-f4be-4941-89d8-7e68d809263b\") " pod="openshift-ingress-canary/ingress-canary-5zsmj" Sep 07 12:53:22.174219 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.174197 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" Sep 07 12:53:22.230759 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.230717 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5zsmj" Sep 07 12:53:22.299928 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.299898 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-6kghh"] Sep 07 12:53:22.304918 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.304893 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-5fzst"] Sep 07 12:53:22.306123 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:22.306103 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4561d14_f779_4e6f_b63c_09f5d09bafed.slice/crio-5d935fa657d4724c530b2de86f43e65ad83aeb6f30c0d88422a9bd2f60de9a64 WatchSource:0}: Error finding container 5d935fa657d4724c530b2de86f43e65ad83aeb6f30c0d88422a9bd2f60de9a64: Status 404 returned error can't find the container with id 5d935fa657d4724c530b2de86f43e65ad83aeb6f30c0d88422a9bd2f60de9a64 Sep 07 12:53:22.306721 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:22.306652 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc212698_c23a_4c52_83b5_137b94537bda.slice/crio-4cec85dd39d73164582478be1091c9b911c9ec79fe7101b2438893d3479dda38 WatchSource:0}: Error finding container 4cec85dd39d73164582478be1091c9b911c9ec79fe7101b2438893d3479dda38: Status 404 returned error can't find the container with id 4cec85dd39d73164582478be1091c9b911c9ec79fe7101b2438893d3479dda38 Sep 07 12:53:22.339541 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.339520 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt"] Sep 07 12:53:22.342778 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:22.342757 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b88887b_d35e_4197_987a_7c91b196cba8.slice/crio-ddd1ce69b5c00b23ffc6146b4cd901d99f04658382155d4bbbd1e4b66a28648c WatchSource:0}: Error finding container ddd1ce69b5c00b23ffc6146b4cd901d99f04658382155d4bbbd1e4b66a28648c: Status 404 returned error can't find the container with id ddd1ce69b5c00b23ffc6146b4cd901d99f04658382155d4bbbd1e4b66a28648c Sep 07 12:53:22.362547 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.362526 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn"] Sep 07 12:53:22.366150 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:22.366103 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73cfc77_cf60_4c83_9d2c_1f5a56779218.slice/crio-976f00d6343b3c309eae8966bf517aea48ae9f0b81afe5315f257b7e8e8a13f5 WatchSource:0}: Error finding container 976f00d6343b3c309eae8966bf517aea48ae9f0b81afe5315f257b7e8e8a13f5: Status 404 returned error can't find the container with id 976f00d6343b3c309eae8966bf517aea48ae9f0b81afe5315f257b7e8e8a13f5 Sep 07 12:53:22.400614 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.400591 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-rmcsw"] Sep 07 12:53:22.402764 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:22.402743 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod703a67a7_558f_450b_9e14_d9c2d8a2109b.slice/crio-0d0c185f0a8890fe8540c4d0bbe377c39a85688c0f8d2ae67c7239a647bf691a WatchSource:0}: Error finding container 0d0c185f0a8890fe8540c4d0bbe377c39a85688c0f8d2ae67c7239a647bf691a: Status 404 returned error can't find the container with id 0d0c185f0a8890fe8540c4d0bbe377c39a85688c0f8d2ae67c7239a647bf691a Sep 07 12:53:22.458251 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.458223 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:22.458404 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.458329 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:22.458404 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.458355 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:22.458516 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.458405 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle podName:f54978ff-9e19-427c-bec3-602c1ed25a1e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:23.458383237 +0000 UTC m=+37.489133001 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle") pod "router-default-6998fdb454-4p474" (UID: "f54978ff-9e19-427c-bec3-602c1ed25a1e") : configmap references non-existent config key: service-ca.crt Sep 07 12:53:22.458516 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.458460 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 12:53:22.458516 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.458463 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 12:53:22.458516 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.458481 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c6bcbc855-x2htj: secret "image-registry-tls" not found Sep 07 12:53:22.458704 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.458504 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs podName:f54978ff-9e19-427c-bec3-602c1ed25a1e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:23.458492947 +0000 UTC m=+37.489242695 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs") pod "router-default-6998fdb454-4p474" (UID: "f54978ff-9e19-427c-bec3-602c1ed25a1e") : secret "router-metrics-certs-default" not found Sep 07 12:53:22.458704 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.458540 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls podName:15ddd367-8b23-4e84-b27f-187d89b0a7bc nodeName:}" failed. No retries permitted until 2026-09-07 12:53:23.458533262 +0000 UTC m=+37.489283011 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls") pod "image-registry-c6bcbc855-x2htj" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc") : secret "image-registry-tls" not found Sep 07 12:53:22.468538 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.468510 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-8nr54"] Sep 07 12:53:22.472108 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:22.472088 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc18ca5b5_32d3_4953_a134_397fce347a66.slice/crio-8493ec7b6ad86e194a198a60323d21651ffd6e189e06fe30755ed35b93af3881 WatchSource:0}: Error finding container 8493ec7b6ad86e194a198a60323d21651ffd6e189e06fe30755ed35b93af3881: Status 404 returned error can't find the container with id 8493ec7b6ad86e194a198a60323d21651ffd6e189e06fe30755ed35b93af3881 Sep 07 12:53:22.476581 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.476567 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:22.478893 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.478876 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 07 12:53:22.520378 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.520352 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5zsmj"] Sep 07 12:53:22.523441 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:22.523424 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd7a4a12_f4be_4941_89d8_7e68d809263b.slice/crio-77fe75812b6a66548a252c68a14e64abd5493244f8ebf9b161f210ee04d9def1 WatchSource:0}: Error finding container 77fe75812b6a66548a252c68a14e64abd5493244f8ebf9b161f210ee04d9def1: Status 404 returned error can't find the container with id 77fe75812b6a66548a252c68a14e64abd5493244f8ebf9b161f210ee04d9def1 Sep 07 12:53:22.559161 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.559142 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:22.559238 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.559172 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-szxpg\" (UID: \"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:22.559285 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.559273 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 07 12:53:22.559285 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.559274 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 12:53:22.559360 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.559306 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:22.559360 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.559321 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls podName:ac7c410e-42b2-4bcd-9099-9b8d7244cc5e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:23.559307586 +0000 UTC m=+37.590057334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-szxpg" (UID: "ac7c410e-42b2-4bcd-9099-9b8d7244cc5e") : secret "samples-operator-tls" not found Sep 07 12:53:22.559360 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.559350 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 07 12:53:22.559481 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.559394 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls podName:9fd689c9-28b9-4a27-a6a5-bf0133374ac8 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:23.559371273 +0000 UTC m=+37.590121031 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls") pod "dns-default-bq8pr" (UID: "9fd689c9-28b9-4a27-a6a5-bf0133374ac8") : secret "dns-default-metrics-tls" not found Sep 07 12:53:22.559481 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.559436 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls podName:5afdf94f-7a4f-483b-9f83-cab863ef2e7b nodeName:}" failed. No retries permitted until 2026-09-07 12:53:23.559423953 +0000 UTC m=+37.590173705 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-n6cnx" (UID: "5afdf94f-7a4f-483b-9f83-cab863ef2e7b") : secret "cluster-monitoring-operator-tls" not found Sep 07 12:53:22.559546 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.559489 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jdjj8\" (UID: \"d5881837-ea29-48f0-865e-40542a004cf5\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:22.559592 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.559582 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 07 12:53:22.559624 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:22.559615 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert podName:d5881837-ea29-48f0-865e-40542a004cf5 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:23.559605026 +0000 UTC m=+37.590354774 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-jdjj8" (UID: "d5881837-ea29-48f0-865e-40542a004cf5") : secret "networking-console-plugin-cert" not found Sep 07 12:53:22.730984 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.730876 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5zsmj" event={"ID":"cd7a4a12-f4be-4941-89d8-7e68d809263b","Type":"ContainerStarted","Data":"77fe75812b6a66548a252c68a14e64abd5493244f8ebf9b161f210ee04d9def1"} Sep 07 12:53:22.732133 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.732109 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" event={"ID":"c73cfc77-cf60-4c83-9d2c-1f5a56779218","Type":"ContainerStarted","Data":"976f00d6343b3c309eae8966bf517aea48ae9f0b81afe5315f257b7e8e8a13f5"} Sep 07 12:53:22.733211 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.733171 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-5fzst" event={"ID":"d4561d14-f779-4e6f-b63c-09f5d09bafed","Type":"ContainerStarted","Data":"5d935fa657d4724c530b2de86f43e65ad83aeb6f30c0d88422a9bd2f60de9a64"} Sep 07 12:53:22.734405 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.734371 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" event={"ID":"c18ca5b5-32d3-4953-a134-397fce347a66","Type":"ContainerStarted","Data":"8493ec7b6ad86e194a198a60323d21651ffd6e189e06fe30755ed35b93af3881"} Sep 07 12:53:22.735422 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.735400 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-rmcsw" event={"ID":"703a67a7-558f-450b-9e14-d9c2d8a2109b","Type":"ContainerStarted","Data":"0d0c185f0a8890fe8540c4d0bbe377c39a85688c0f8d2ae67c7239a647bf691a"} Sep 07 12:53:22.738489 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.738457 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6dj5g" event={"ID":"67ad982a-f807-4a65-b78d-8427d008e9cd","Type":"ContainerStarted","Data":"a7556f9dd0e286e94eae5a870829581d6af688eeeb90a64035341900f17a59c4"} Sep 07 12:53:22.739634 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.739616 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-6kghh" event={"ID":"dc212698-c23a-4c52-83b5-137b94537bda","Type":"ContainerStarted","Data":"4cec85dd39d73164582478be1091c9b911c9ec79fe7101b2438893d3479dda38"} Sep 07 12:53:22.740644 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.740627 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" event={"ID":"7b88887b-d35e-4197-987a-7c91b196cba8","Type":"ContainerStarted","Data":"ddd1ce69b5c00b23ffc6146b4cd901d99f04658382155d4bbbd1e4b66a28648c"} Sep 07 12:53:22.765285 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.765237 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6dj5g" podStartSLOduration=5.368126962 podStartE2EDuration="35.765226076s" totalImagesPullingTime="30.397099114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:22.763742649 +0000 UTC m=+36.794492430" watchObservedRunningTime="2026-09-07 12:53:22.765226076 +0000 UTC m=+36.795975846" Sep 07 12:53:22.920034 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:22.920001 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:23.012940 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.012834 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:23.469833 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.468708 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:23.469833 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.469005 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:23.469833 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.469073 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:23.469833 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.469289 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle podName:f54978ff-9e19-427c-bec3-602c1ed25a1e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:25.46927016 +0000 UTC m=+39.500019914 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle") pod "router-default-6998fdb454-4p474" (UID: "f54978ff-9e19-427c-bec3-602c1ed25a1e") : configmap references non-existent config key: service-ca.crt Sep 07 12:53:23.469833 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.469668 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 12:53:23.469833 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.469707 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs podName:f54978ff-9e19-427c-bec3-602c1ed25a1e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:25.469694363 +0000 UTC m=+39.500444112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs") pod "router-default-6998fdb454-4p474" (UID: "f54978ff-9e19-427c-bec3-602c1ed25a1e") : secret "router-metrics-certs-default" not found Sep 07 12:53:23.469833 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.469750 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 12:53:23.469833 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.469762 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c6bcbc855-x2htj: secret "image-registry-tls" not found Sep 07 12:53:23.469833 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.469796 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls podName:15ddd367-8b23-4e84-b27f-187d89b0a7bc nodeName:}" failed. No retries permitted until 2026-09-07 12:53:25.46978422 +0000 UTC m=+39.500534028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls") pod "image-registry-c6bcbc855-x2htj" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc") : secret "image-registry-tls" not found Sep 07 12:53:23.478048 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.476855 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:23.478048 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.477403 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:23.481462 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.481433 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnfpv\"" Sep 07 12:53:23.482258 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.481658 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 07 12:53:23.482258 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.481862 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xfxxc\"" Sep 07 12:53:23.569889 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.569855 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:23.570074 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.569938 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jdjj8\" (UID: \"d5881837-ea29-48f0-865e-40542a004cf5\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:23.570074 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.570058 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:23.570167 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.570089 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-szxpg\" (UID: \"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:23.570847 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.570231 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 07 12:53:23.570847 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.570297 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls podName:ac7c410e-42b2-4bcd-9099-9b8d7244cc5e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:25.570277419 +0000 UTC m=+39.601027194 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-szxpg" (UID: "ac7c410e-42b2-4bcd-9099-9b8d7244cc5e") : secret "samples-operator-tls" not found Sep 07 12:53:23.570847 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.570401 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 07 12:53:23.570847 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.570440 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls podName:5afdf94f-7a4f-483b-9f83-cab863ef2e7b nodeName:}" failed. No retries permitted until 2026-09-07 12:53:25.57042642 +0000 UTC m=+39.601176172 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-n6cnx" (UID: "5afdf94f-7a4f-483b-9f83-cab863ef2e7b") : secret "cluster-monitoring-operator-tls" not found Sep 07 12:53:23.570847 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.570499 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 07 12:53:23.570847 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.570532 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert podName:d5881837-ea29-48f0-865e-40542a004cf5 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:25.57052158 +0000 UTC m=+39.601271332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-jdjj8" (UID: "d5881837-ea29-48f0-865e-40542a004cf5") : secret "networking-console-plugin-cert" not found Sep 07 12:53:23.570847 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.570590 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 12:53:23.570847 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:23.570617 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls podName:9fd689c9-28b9-4a27-a6a5-bf0133374ac8 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:25.570608319 +0000 UTC m=+39.601358081 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls") pod "dns-default-bq8pr" (UID: "9fd689c9-28b9-4a27-a6a5-bf0133374ac8") : secret "dns-default-metrics-tls" not found Sep 07 12:53:23.835232 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.835153 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:23.947054 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:23.947033 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:24.072340 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:24.072241 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:24.749093 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:24.749061 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-6kghh" event={"ID":"dc212698-c23a-4c52-83b5-137b94537bda","Type":"ContainerStarted","Data":"67f0de1dad088b2132822adbed36df414a6d7b106ca57548fff73798efd2e4b8"} Sep 07 12:53:24.783358 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:24.783291 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-6kghh" podStartSLOduration=9.226436073 podStartE2EDuration="10.783275209s" totalImagesPullingTime="1.556839136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:24.782692026 +0000 UTC m=+38.813441813" watchObservedRunningTime="2026-09-07 12:53:24.783275209 +0000 UTC m=+38.814024981" Sep 07 12:53:24.900616 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:24.900583 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:24.976781 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:24.976753 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:25.486702 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:25.486621 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:25.486702 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:25.486683 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:25.486898 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:25.486747 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:25.486898 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.486809 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 12:53:25.486898 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.486886 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 12:53:25.487108 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.486904 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c6bcbc855-x2htj: secret "image-registry-tls" not found Sep 07 12:53:25.487108 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.486890 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs podName:f54978ff-9e19-427c-bec3-602c1ed25a1e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:29.486866682 +0000 UTC m=+43.517616432 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs") pod "router-default-6998fdb454-4p474" (UID: "f54978ff-9e19-427c-bec3-602c1ed25a1e") : secret "router-metrics-certs-default" not found Sep 07 12:53:25.487108 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.486961 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle podName:f54978ff-9e19-427c-bec3-602c1ed25a1e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:29.486946396 +0000 UTC m=+43.517696144 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle") pod "router-default-6998fdb454-4p474" (UID: "f54978ff-9e19-427c-bec3-602c1ed25a1e") : configmap references non-existent config key: service-ca.crt Sep 07 12:53:25.487108 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.487013 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls podName:15ddd367-8b23-4e84-b27f-187d89b0a7bc nodeName:}" failed. No retries permitted until 2026-09-07 12:53:29.487006088 +0000 UTC m=+43.517755836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls") pod "image-registry-c6bcbc855-x2htj" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc") : secret "image-registry-tls" not found Sep 07 12:53:25.587800 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:25.587675 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:25.587800 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:25.587730 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-szxpg\" (UID: \"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:25.587800 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:25.587787 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:25.588278 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:25.587848 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jdjj8\" (UID: \"d5881837-ea29-48f0-865e-40542a004cf5\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:25.588278 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.588029 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 07 12:53:25.588278 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.588097 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert podName:d5881837-ea29-48f0-865e-40542a004cf5 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:29.588078327 +0000 UTC m=+43.618828079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-jdjj8" (UID: "d5881837-ea29-48f0-865e-40542a004cf5") : secret "networking-console-plugin-cert" not found Sep 07 12:53:25.588278 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.588096 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 12:53:25.588278 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.588139 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 07 12:53:25.588278 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.588152 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls podName:9fd689c9-28b9-4a27-a6a5-bf0133374ac8 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:29.588139213 +0000 UTC m=+43.618888961 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls") pod "dns-default-bq8pr" (UID: "9fd689c9-28b9-4a27-a6a5-bf0133374ac8") : secret "dns-default-metrics-tls" not found Sep 07 12:53:25.588278 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.588097 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 07 12:53:25.588278 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.588186 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls podName:ac7c410e-42b2-4bcd-9099-9b8d7244cc5e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:29.588178647 +0000 UTC m=+43.618928395 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-szxpg" (UID: "ac7c410e-42b2-4bcd-9099-9b8d7244cc5e") : secret "samples-operator-tls" not found Sep 07 12:53:25.588278 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:25.588199 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls podName:5afdf94f-7a4f-483b-9f83-cab863ef2e7b nodeName:}" failed. No retries permitted until 2026-09-07 12:53:29.588190858 +0000 UTC m=+43.618940607 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-n6cnx" (UID: "5afdf94f-7a4f-483b-9f83-cab863ef2e7b") : secret "cluster-monitoring-operator-tls" not found Sep 07 12:53:25.910775 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:25.910732 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:25.985020 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:25.984962 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:26.566682 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:26.566644 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:26.637692 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:26.637640 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:27.693058 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:27.693020 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:28.769768 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:28.769739 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" event={"ID":"c73cfc77-cf60-4c83-9d2c-1f5a56779218","Type":"ContainerStarted","Data":"eef58704faa3ff73b34113c7960898b3e9993b1a868324792141d43fc73ecd8a"} Sep 07 12:53:28.771668 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:28.771647 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/0.log" Sep 07 12:53:28.771773 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:28.771687 2576 generic.go:356] "Generic (PLEG): container finished" podID="d4561d14-f779-4e6f-b63c-09f5d09bafed" containerID="0b8154df77bc169a212815f708f823014e8ffc84df8d91d3a2f7330e780166f7" exitCode=255 Sep 07 12:53:28.771773 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:28.771744 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-5fzst" event={"ID":"d4561d14-f779-4e6f-b63c-09f5d09bafed","Type":"ContainerDied","Data":"0b8154df77bc169a212815f708f823014e8ffc84df8d91d3a2f7330e780166f7"} Sep 07 12:53:28.771989 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:28.771951 2576 scope.go:117] "RemoveContainer" containerID="0b8154df77bc169a212815f708f823014e8ffc84df8d91d3a2f7330e780166f7" Sep 07 12:53:28.775001 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:28.774959 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" event={"ID":"c18ca5b5-32d3-4953-a134-397fce347a66","Type":"ContainerStarted","Data":"dcc44219299f3655416b63b9f67146b9b8b37b83c2cf21c756f90e80cdbb3054"} Sep 07 12:53:28.776580 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:28.776560 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" event={"ID":"7b88887b-d35e-4197-987a-7c91b196cba8","Type":"ContainerStarted","Data":"196f04fe63838f49709b05925fe8036a18e747626568f34dfe5df26ab1f80927"} Sep 07 12:53:28.787766 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:28.787614 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" podStartSLOduration=8.722800057 podStartE2EDuration="14.787596725s" totalImagesPullingTime="6.064796668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:28.787262507 +0000 UTC m=+42.818012295" watchObservedRunningTime="2026-09-07 12:53:28.787596725 +0000 UTC m=+42.818346501" Sep 07 12:53:28.811342 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:28.810281 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" podStartSLOduration=8.851803037 podStartE2EDuration="14.810265305s" totalImagesPullingTime="5.958462268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:28.809220625 +0000 UTC m=+42.839970412" watchObservedRunningTime="2026-09-07 12:53:28.810265305 +0000 UTC m=+42.841015123" Sep 07 12:53:28.862684 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:28.862612 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" podStartSLOduration=8.774306245 podStartE2EDuration="14.862591958s" totalImagesPullingTime="6.088285713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:28.862441718 +0000 UTC m=+42.893191489" watchObservedRunningTime="2026-09-07 12:53:28.862591958 +0000 UTC m=+42.893341730" Sep 07 12:53:29.531089 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.531046 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:29.531249 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.531152 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:29.531249 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.531177 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:29.531369 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.531284 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 12:53:29.531369 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.531294 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c6bcbc855-x2htj: secret "image-registry-tls" not found Sep 07 12:53:29.531369 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.531337 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls podName:15ddd367-8b23-4e84-b27f-187d89b0a7bc nodeName:}" failed. No retries permitted until 2026-09-07 12:53:37.531324254 +0000 UTC m=+51.562074004 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls") pod "image-registry-c6bcbc855-x2htj" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc") : secret "image-registry-tls" not found Sep 07 12:53:29.531625 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.531611 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle podName:f54978ff-9e19-427c-bec3-602c1ed25a1e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:37.531601375 +0000 UTC m=+51.562351138 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle") pod "router-default-6998fdb454-4p474" (UID: "f54978ff-9e19-427c-bec3-602c1ed25a1e") : configmap references non-existent config key: service-ca.crt Sep 07 12:53:29.531683 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.531664 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 12:53:29.531734 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.531684 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs podName:f54978ff-9e19-427c-bec3-602c1ed25a1e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:37.531678499 +0000 UTC m=+51.562428247 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs") pod "router-default-6998fdb454-4p474" (UID: "f54978ff-9e19-427c-bec3-602c1ed25a1e") : secret "router-metrics-certs-default" not found Sep 07 12:53:29.632177 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.632137 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:29.632318 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.632186 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-szxpg\" (UID: \"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:29.632318 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.632243 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:29.632318 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.632302 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jdjj8\" (UID: \"d5881837-ea29-48f0-865e-40542a004cf5\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:29.632431 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.632418 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 07 12:53:29.632490 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.632481 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert podName:d5881837-ea29-48f0-865e-40542a004cf5 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:37.632463472 +0000 UTC m=+51.663213224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-jdjj8" (UID: "d5881837-ea29-48f0-865e-40542a004cf5") : secret "networking-console-plugin-cert" not found Sep 07 12:53:29.632850 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.632837 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 12:53:29.632892 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.632883 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls podName:9fd689c9-28b9-4a27-a6a5-bf0133374ac8 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:37.632870028 +0000 UTC m=+51.663619780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls") pod "dns-default-bq8pr" (UID: "9fd689c9-28b9-4a27-a6a5-bf0133374ac8") : secret "dns-default-metrics-tls" not found Sep 07 12:53:29.632948 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.632939 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 07 12:53:29.633024 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.632998 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls podName:ac7c410e-42b2-4bcd-9099-9b8d7244cc5e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:37.632963115 +0000 UTC m=+51.663712870 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-szxpg" (UID: "ac7c410e-42b2-4bcd-9099-9b8d7244cc5e") : secret "samples-operator-tls" not found Sep 07 12:53:29.633069 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.633051 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 07 12:53:29.633105 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.633086 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls podName:5afdf94f-7a4f-483b-9f83-cab863ef2e7b nodeName:}" failed. No retries permitted until 2026-09-07 12:53:37.633076934 +0000 UTC m=+51.663826691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-n6cnx" (UID: "5afdf94f-7a4f-483b-9f83-cab863ef2e7b") : secret "cluster-monitoring-operator-tls" not found Sep 07 12:53:29.687685 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.687644 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:29.703215 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.703184 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-56p4x"] Sep 07 12:53:29.721884 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.721851 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-56p4x"] Sep 07 12:53:29.722053 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.722004 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-56p4x" Sep 07 12:53:29.723898 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.723874 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 07 12:53:29.724336 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.724315 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 07 12:53:29.724591 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.724574 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zmrqk\"" Sep 07 12:53:29.782689 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.782615 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/1.log" Sep 07 12:53:29.783455 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.783254 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/0.log" Sep 07 12:53:29.783455 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.783281 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:29.783455 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.783305 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-5fzst" event={"ID":"d4561d14-f779-4e6f-b63c-09f5d09bafed","Type":"ContainerDied","Data":"b80cdf6fdac2f734cf5098730923571c9cd3a97a1808d5d5efda21a72e576149"} Sep 07 12:53:29.783455 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.783287 2576 generic.go:356] "Generic (PLEG): container finished" podID="d4561d14-f779-4e6f-b63c-09f5d09bafed" containerID="b80cdf6fdac2f734cf5098730923571c9cd3a97a1808d5d5efda21a72e576149" exitCode=255 Sep 07 12:53:29.783455 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.783350 2576 scope.go:117] "RemoveContainer" containerID="0b8154df77bc169a212815f708f823014e8ffc84df8d91d3a2f7330e780166f7" Sep 07 12:53:29.783768 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.783752 2576 scope.go:117] "RemoveContainer" containerID="b80cdf6fdac2f734cf5098730923571c9cd3a97a1808d5d5efda21a72e576149" Sep 07 12:53:29.783935 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:29.783915 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-5fzst_openshift-console-operator(d4561d14-f779-4e6f-b63c-09f5d09bafed)\"" pod="openshift-console-operator/console-operator-85b454874b-5fzst" podUID="d4561d14-f779-4e6f-b63c-09f5d09bafed" Sep 07 12:53:29.788378 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.788305 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-rmcsw" event={"ID":"703a67a7-558f-450b-9e14-d9c2d8a2109b","Type":"ContainerStarted","Data":"26f113c1c435827fa7222bfe6d94653641bdf85d0eb86bcb7883dc03dcab51c0"} Sep 07 12:53:29.816804 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.816754 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-rmcsw" podStartSLOduration=9.455095187 podStartE2EDuration="15.816737382s" totalImagesPullingTime="6.361642195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:29.816285643 +0000 UTC m=+43.847035411" watchObservedRunningTime="2026-09-07 12:53:29.816737382 +0000 UTC m=+43.847487153" Sep 07 12:53:29.835741 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.835714 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49hnj\" (UniqueName: \"kubernetes.io/projected/8709830e-e318-4149-8bd1-6d807c996d1b-kube-api-access-49hnj\") pod \"migrator-fdbb8fdff-56p4x\" (UID: \"8709830e-e318-4149-8bd1-6d807c996d1b\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-56p4x" Sep 07 12:53:29.883414 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.883383 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:29.937038 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.936951 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49hnj\" (UniqueName: \"kubernetes.io/projected/8709830e-e318-4149-8bd1-6d807c996d1b-kube-api-access-49hnj\") pod \"migrator-fdbb8fdff-56p4x\" (UID: \"8709830e-e318-4149-8bd1-6d807c996d1b\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-56p4x" Sep 07 12:53:29.947687 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:29.947636 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49hnj\" (UniqueName: \"kubernetes.io/projected/8709830e-e318-4149-8bd1-6d807c996d1b-kube-api-access-49hnj\") pod \"migrator-fdbb8fdff-56p4x\" (UID: \"8709830e-e318-4149-8bd1-6d807c996d1b\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-56p4x" Sep 07 12:53:30.024596 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.024554 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:30.035382 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.035310 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-56p4x" Sep 07 12:53:30.123373 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.123210 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:30.206008 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.205958 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4b89k"] Sep 07 12:53:30.224255 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.224220 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:30.239010 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.238956 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4b89k"] Sep 07 12:53:30.239179 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.239114 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.241075 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.240845 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 07 12:53:30.241075 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.240865 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6fwj5\"" Sep 07 12:53:30.241348 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.241321 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 07 12:53:30.342003 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.341877 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.342003 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.341927 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/656a68ec-df05-470c-8d6e-756773871446-crio-socket\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.342269 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.342072 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/656a68ec-df05-470c-8d6e-756773871446-data-volume\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.342269 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.342123 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.342269 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.342203 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qpds\" (UniqueName: \"kubernetes.io/projected/656a68ec-df05-470c-8d6e-756773871446-kube-api-access-9qpds\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.443039 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.442999 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.443211 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.443066 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/656a68ec-df05-470c-8d6e-756773871446-crio-socket\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.443211 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:30.443076 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 12:53:30.443211 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.443124 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/656a68ec-df05-470c-8d6e-756773871446-data-volume\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.443211 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:30.443169 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm podName:656a68ec-df05-470c-8d6e-756773871446 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:30.943146204 +0000 UTC m=+44.973895957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4b89k" (UID: "656a68ec-df05-470c-8d6e-756773871446") : configmap "kube-rbac-proxy" not found Sep 07 12:53:30.443211 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.443195 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.443473 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.443290 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qpds\" (UniqueName: \"kubernetes.io/projected/656a68ec-df05-470c-8d6e-756773871446-kube-api-access-9qpds\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.443473 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.443299 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/656a68ec-df05-470c-8d6e-756773871446-crio-socket\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.443473 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:30.443385 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 12:53:30.443810 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:30.443791 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls podName:656a68ec-df05-470c-8d6e-756773871446 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:30.943772294 +0000 UTC m=+44.974522044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4b89k" (UID: "656a68ec-df05-470c-8d6e-756773871446") : secret "insights-runtime-extractor-tls" not found Sep 07 12:53:30.444151 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.444132 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/656a68ec-df05-470c-8d6e-756773871446-data-volume\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.453993 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.453950 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qpds\" (UniqueName: \"kubernetes.io/projected/656a68ec-df05-470c-8d6e-756773871446-kube-api-access-9qpds\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.485345 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.485319 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:30.585747 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.585708 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:30.747278 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.747126 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:30.794131 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.794097 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/1.log" Sep 07 12:53:30.794850 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.794824 2576 scope.go:117] "RemoveContainer" containerID="b80cdf6fdac2f734cf5098730923571c9cd3a97a1808d5d5efda21a72e576149" Sep 07 12:53:30.795008 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:30.794965 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-5fzst_openshift-console-operator(d4561d14-f779-4e6f-b63c-09f5d09bafed)\"" pod="openshift-console-operator/console-operator-85b454874b-5fzst" podUID="d4561d14-f779-4e6f-b63c-09f5d09bafed" Sep 07 12:53:30.818795 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.818765 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:30.888631 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.888602 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:30.947879 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.947838 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.948040 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:30.947931 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:30.948040 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:30.947995 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 12:53:30.948141 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:30.948074 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm podName:656a68ec-df05-470c-8d6e-756773871446 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:31.948049064 +0000 UTC m=+45.978798857 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4b89k" (UID: "656a68ec-df05-470c-8d6e-756773871446") : configmap "kube-rbac-proxy" not found Sep 07 12:53:30.948306 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:30.948216 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 12:53:30.948414 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:30.948316 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls podName:656a68ec-df05-470c-8d6e-756773871446 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:31.948294389 +0000 UTC m=+45.979044152 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4b89k" (UID: "656a68ec-df05-470c-8d6e-756773871446") : secret "insights-runtime-extractor-tls" not found Sep 07 12:53:31.199153 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:31.199123 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-56p4x"] Sep 07 12:53:31.291782 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:31.291751 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8709830e_e318_4149_8bd1_6d807c996d1b.slice/crio-c332cb19c86b63941c7082776984d0272e9af7e600d97727b94879124dd180e8 WatchSource:0}: Error finding container c332cb19c86b63941c7082776984d0272e9af7e600d97727b94879124dd180e8: Status 404 returned error can't find the container with id c332cb19c86b63941c7082776984d0272e9af7e600d97727b94879124dd180e8 Sep 07 12:53:31.799484 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:31.799449 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-56p4x" event={"ID":"8709830e-e318-4149-8bd1-6d807c996d1b","Type":"ContainerStarted","Data":"c332cb19c86b63941c7082776984d0272e9af7e600d97727b94879124dd180e8"} Sep 07 12:53:31.801079 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:31.801053 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5zsmj" event={"ID":"cd7a4a12-f4be-4941-89d8-7e68d809263b","Type":"ContainerStarted","Data":"cc4da86ce64f66f8c3d87e3d58fbe1608ecf196a5af07c80107982fc5fdffcc8"} Sep 07 12:53:31.819100 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:31.819049 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5zsmj" podStartSLOduration=2.022039033 podStartE2EDuration="10.819033328s" totalImagesPullingTime="8.796994295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:31.81882818 +0000 UTC m=+45.849577950" watchObservedRunningTime="2026-09-07 12:53:31.819033328 +0000 UTC m=+45.849783139" Sep 07 12:53:31.942988 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:31.942940 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2lm8h_a7296228-d10d-43e8-9c48-d6d72e58532a/dns-node-resolver/0.log" Sep 07 12:53:31.956409 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:31.956371 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:31.956664 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:31.956439 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:31.956664 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:31.956525 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 12:53:31.956664 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:31.956542 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 12:53:31.956664 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:31.956574 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls podName:656a68ec-df05-470c-8d6e-756773871446 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:33.956561712 +0000 UTC m=+47.987311460 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4b89k" (UID: "656a68ec-df05-470c-8d6e-756773871446") : secret "insights-runtime-extractor-tls" not found Sep 07 12:53:31.956664 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:31.956601 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm podName:656a68ec-df05-470c-8d6e-756773871446 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:33.95658476 +0000 UTC m=+47.987334526 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4b89k" (UID: "656a68ec-df05-470c-8d6e-756773871446") : configmap "kube-rbac-proxy" not found Sep 07 12:53:32.004889 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:32.004844 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:32.005035 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:32.004915 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:32.005303 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:32.005289 2576 scope.go:117] "RemoveContainer" containerID="b80cdf6fdac2f734cf5098730923571c9cd3a97a1808d5d5efda21a72e576149" Sep 07 12:53:32.005473 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:32.005455 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-5fzst_openshift-console-operator(d4561d14-f779-4e6f-b63c-09f5d09bafed)\"" pod="openshift-console-operator/console-operator-85b454874b-5fzst" podUID="d4561d14-f779-4e6f-b63c-09f5d09bafed" Sep 07 12:53:32.510718 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:32.510688 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:32.608777 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:32.608721 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:33.343088 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:33.343018 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mmqsg_8840b2d7-5f57-4887-9d76-1966edbec149/node-ca/0.log" Sep 07 12:53:33.417853 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:33.417819 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:33.743543 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:33.743479 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5zsmj_cd7a4a12-f4be-4941-89d8-7e68d809263b/serve-healthcheck-canary/0.log" Sep 07 12:53:33.807817 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:33.807781 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-56p4x" event={"ID":"8709830e-e318-4149-8bd1-6d807c996d1b","Type":"ContainerStarted","Data":"1d3ced25c5e2575f238b2a856d940cc3f50d06ec52cdac4151c4dce4366c6262"} Sep 07 12:53:33.808013 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:33.807881 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-56p4x" event={"ID":"8709830e-e318-4149-8bd1-6d807c996d1b","Type":"ContainerStarted","Data":"da7438a3662ef68c7b6092f005887e422d985873a5faa4049ffe526c1638cdc5"} Sep 07 12:53:33.826907 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:33.826863 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-56p4x" podStartSLOduration=3.069835974 podStartE2EDuration="4.826848234s" totalImagesPullingTime="1.75701226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:33.826360335 +0000 UTC m=+47.857110121" watchObservedRunningTime="2026-09-07 12:53:33.826848234 +0000 UTC m=+47.857598005" Sep 07 12:53:33.944193 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:33.944167 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-56p4x_8709830e-e318-4149-8bd1-6d807c996d1b/migrator/0.log" Sep 07 12:53:33.975188 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:33.975155 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:33.975380 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:33.975240 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:33.975380 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:33.975269 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 12:53:33.975380 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:33.975325 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm podName:656a68ec-df05-470c-8d6e-756773871446 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:37.975311394 +0000 UTC m=+52.006061143 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4b89k" (UID: "656a68ec-df05-470c-8d6e-756773871446") : configmap "kube-rbac-proxy" not found Sep 07 12:53:33.975380 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:33.975371 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 12:53:33.975560 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:33.975430 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls podName:656a68ec-df05-470c-8d6e-756773871446 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:37.975414391 +0000 UTC m=+52.006164167 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4b89k" (UID: "656a68ec-df05-470c-8d6e-756773871446") : secret "insights-runtime-extractor-tls" not found Sep 07 12:53:34.143454 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:34.143421 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-56p4x_8709830e-e318-4149-8bd1-6d807c996d1b/graceful-termination/0.log" Sep 07 12:53:34.345066 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:34.345039 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-9r4xn_c73cfc77-cf60-4c83-9d2c-1f5a56779218/kube-storage-version-migrator-operator/0.log" Sep 07 12:53:34.358944 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:34.358915 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:34.457367 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:34.457278 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:36.201174 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:36.201139 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:37.383366 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:37.383333 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:37.606484 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:37.606455 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:37.606647 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:37.606535 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:37.606647 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:37.606595 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:37.606717 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.606653 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 12:53:37.606757 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.606723 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle podName:f54978ff-9e19-427c-bec3-602c1ed25a1e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:53.606704285 +0000 UTC m=+67.637454048 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle") pod "router-default-6998fdb454-4p474" (UID: "f54978ff-9e19-427c-bec3-602c1ed25a1e") : configmap references non-existent config key: service-ca.crt Sep 07 12:53:37.606757 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.606743 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 12:53:37.606829 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.606759 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c6bcbc855-x2htj: secret "image-registry-tls" not found Sep 07 12:53:37.606829 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.606747 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs podName:f54978ff-9e19-427c-bec3-602c1ed25a1e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:53.606738458 +0000 UTC m=+67.637488213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs") pod "router-default-6998fdb454-4p474" (UID: "f54978ff-9e19-427c-bec3-602c1ed25a1e") : secret "router-metrics-certs-default" not found Sep 07 12:53:37.606892 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.606861 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls podName:15ddd367-8b23-4e84-b27f-187d89b0a7bc nodeName:}" failed. No retries permitted until 2026-09-07 12:53:53.606849956 +0000 UTC m=+67.637599706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls") pod "image-registry-c6bcbc855-x2htj" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc") : secret "image-registry-tls" not found Sep 07 12:53:37.707892 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:37.707807 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:37.707892 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:37.707884 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jdjj8\" (UID: \"d5881837-ea29-48f0-865e-40542a004cf5\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:37.708165 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.707988 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 07 12:53:37.708165 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.708033 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 07 12:53:37.708165 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.708045 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls podName:5afdf94f-7a4f-483b-9f83-cab863ef2e7b nodeName:}" failed. No retries permitted until 2026-09-07 12:53:53.708030217 +0000 UTC m=+67.738779966 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-n6cnx" (UID: "5afdf94f-7a4f-483b-9f83-cab863ef2e7b") : secret "cluster-monitoring-operator-tls" not found Sep 07 12:53:37.708165 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.708102 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert podName:d5881837-ea29-48f0-865e-40542a004cf5 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:53.708089406 +0000 UTC m=+67.738839156 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-jdjj8" (UID: "d5881837-ea29-48f0-865e-40542a004cf5") : secret "networking-console-plugin-cert" not found Sep 07 12:53:37.708165 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:37.708037 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:37.708165 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:37.708149 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-szxpg\" (UID: \"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:37.708357 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.708103 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 12:53:37.708357 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.708246 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls podName:9fd689c9-28b9-4a27-a6a5-bf0133374ac8 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:53.70822919 +0000 UTC m=+67.738978943 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls") pod "dns-default-bq8pr" (UID: "9fd689c9-28b9-4a27-a6a5-bf0133374ac8") : secret "dns-default-metrics-tls" not found Sep 07 12:53:37.708357 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.708262 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 07 12:53:37.708357 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:37.708309 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls podName:ac7c410e-42b2-4bcd-9099-9b8d7244cc5e nodeName:}" failed. No retries permitted until 2026-09-07 12:53:53.708298864 +0000 UTC m=+67.739048642 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-szxpg" (UID: "ac7c410e-42b2-4bcd-9099-9b8d7244cc5e") : secret "samples-operator-tls" not found Sep 07 12:53:38.010160 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:38.010082 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:38.010293 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:38.010168 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:38.010331 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:38.010304 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 12:53:38.010375 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:38.010364 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls podName:656a68ec-df05-470c-8d6e-756773871446 nodeName:}" failed. No retries permitted until 2026-09-07 12:53:46.010350149 +0000 UTC m=+60.041099897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4b89k" (UID: "656a68ec-df05-470c-8d6e-756773871446") : secret "insights-runtime-extractor-tls" not found Sep 07 12:53:38.010643 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:38.010615 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/656a68ec-df05-470c-8d6e-756773871446-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:38.833161 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:38.833130 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:38.934704 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:38.934667 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:39.121953 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:39.121868 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:39.200310 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:39.200282 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:39.699608 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:39.699580 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:40.304211 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:40.304176 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:40.512404 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:40.512375 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:40.907620 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:40.907593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:41.498540 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:41.498510 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:41.602772 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:41.602744 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:41.691090 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:41.691059 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:41.901009 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:41.900962 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:42.099250 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:42.099209 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:42.517320 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:42.517258 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:42.595444 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:42.595404 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:43.297569 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:43.297534 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:43.357103 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:43.357071 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:43.696493 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:43.696410 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:43.893356 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:43.893325 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:44.132001 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:44.131953 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:44.235640 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:44.235600 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:44.336862 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:44.336833 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:44.402863 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:44.402782 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:45.096140 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:45.096105 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:45.298572 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:45.298538 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:45.506515 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:45.506426 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:45.899679 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:45.899645 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:46.078047 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:46.078016 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:46.080442 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:46.080416 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/656a68ec-df05-470c-8d6e-756773871446-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4b89k\" (UID: \"656a68ec-df05-470c-8d6e-756773871446\") " pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:46.094590 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:46.094562 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:46.151749 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:46.151693 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4b89k" Sep 07 12:53:46.425892 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:46.425824 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4b89k"] Sep 07 12:53:46.428614 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:46.428592 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod656a68ec_df05_470c_8d6e_756773871446.slice/crio-adfb39f9fc76f08c5caa39a085e656fc51cc3481a4b514e1a1319714f4a2a166 WatchSource:0}: Error finding container adfb39f9fc76f08c5caa39a085e656fc51cc3481a4b514e1a1319714f4a2a166: Status 404 returned error can't find the container with id adfb39f9fc76f08c5caa39a085e656fc51cc3481a4b514e1a1319714f4a2a166 Sep 07 12:53:46.497244 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:46.497215 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:46.719823 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:46.719754 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lpc6s" Sep 07 12:53:46.848232 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:46.848199 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4b89k" event={"ID":"656a68ec-df05-470c-8d6e-756773871446","Type":"ContainerStarted","Data":"1153c54fc7a6a283596ca65a133ecf892fffa067caa974f5cde5fd23b05e215b"} Sep 07 12:53:46.848373 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:46.848249 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4b89k" event={"ID":"656a68ec-df05-470c-8d6e-756773871446","Type":"ContainerStarted","Data":"adfb39f9fc76f08c5caa39a085e656fc51cc3481a4b514e1a1319714f4a2a166"} Sep 07 12:53:47.109933 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:47.109908 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:47.476305 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:47.476229 2576 scope.go:117] "RemoveContainer" containerID="b80cdf6fdac2f734cf5098730923571c9cd3a97a1808d5d5efda21a72e576149" Sep 07 12:53:47.508171 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:47.508147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:47.738844 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:47.738753 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:47.853797 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:47.853773 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 12:53:47.854314 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:47.854295 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/1.log" Sep 07 12:53:47.854434 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:47.854330 2576 generic.go:356] "Generic (PLEG): container finished" podID="d4561d14-f779-4e6f-b63c-09f5d09bafed" containerID="86922340083aeb2cb8765db6af4d3823a55cd2c2d7fce7a2a5bec795d46d6359" exitCode=255 Sep 07 12:53:47.854434 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:47.854415 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-5fzst" event={"ID":"d4561d14-f779-4e6f-b63c-09f5d09bafed","Type":"ContainerDied","Data":"86922340083aeb2cb8765db6af4d3823a55cd2c2d7fce7a2a5bec795d46d6359"} Sep 07 12:53:47.854527 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:47.854489 2576 scope.go:117] "RemoveContainer" containerID="b80cdf6fdac2f734cf5098730923571c9cd3a97a1808d5d5efda21a72e576149" Sep 07 12:53:47.854893 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:47.854863 2576 scope.go:117] "RemoveContainer" containerID="86922340083aeb2cb8765db6af4d3823a55cd2c2d7fce7a2a5bec795d46d6359" Sep 07 12:53:47.855118 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:47.855095 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-5fzst_openshift-console-operator(d4561d14-f779-4e6f-b63c-09f5d09bafed)\"" pod="openshift-console-operator/console-operator-85b454874b-5fzst" podUID="d4561d14-f779-4e6f-b63c-09f5d09bafed" Sep 07 12:53:47.858269 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:47.858231 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4b89k" event={"ID":"656a68ec-df05-470c-8d6e-756773871446","Type":"ContainerStarted","Data":"c1cd6ac169d513ae97729fbbfef61e50bee05f6babf142b0ec5c6bc0b5158b27"} Sep 07 12:53:48.313133 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:48.313082 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:48.691211 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:48.691122 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:48.863445 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:48.863420 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 12:53:48.865404 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:48.865381 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4b89k" event={"ID":"656a68ec-df05-470c-8d6e-756773871446","Type":"ContainerStarted","Data":"aad5621957772dafc6b11fcc85d81dcf6fd403c81f877810ac137de70ad157bb"} Sep 07 12:53:48.885221 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:48.885179 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4b89k" podStartSLOduration=16.998606143 podStartE2EDuration="18.885165854s" totalImagesPullingTime="1.886559711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:53:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:48.884423021 +0000 UTC m=+62.915172792" watchObservedRunningTime="2026-09-07 12:53:48.885165854 +0000 UTC m=+62.915915624" Sep 07 12:53:49.096758 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:49.096718 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:49.308289 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:49.308255 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:49.373291 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:49.373168 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:49.434463 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:49.434429 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:49.500466 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:49.500438 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:51.216549 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.216517 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:51.218350 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.218330 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 07 12:53:51.228851 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.228832 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6e766773-bcc2-4cb6-8df6-3dc08e9d59cc-original-pull-secret\") pod \"global-pull-secret-syncer-hpq9f\" (UID: \"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc\") " pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:51.286187 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.286163 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hpq9f" Sep 07 12:53:51.317237 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.317220 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs\") pod \"network-metrics-daemon-pkv6v\" (UID: \"da204f77-d893-4332-baab-4418cb7e72bc\") " pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:51.319360 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.319345 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 07 12:53:51.330454 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.330430 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da204f77-d893-4332-baab-4418cb7e72bc-metrics-certs\") pod \"network-metrics-daemon-pkv6v\" (UID: \"da204f77-d893-4332-baab-4418cb7e72bc\") " pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:51.410382 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.410355 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnfpv\"" Sep 07 12:53:51.418008 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.417987 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtvwt\" (UniqueName: \"kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt\") pod \"network-check-target-m9c7v\" (UID: \"7b8afbb2-9557-4821-8712-89cc16f472f4\") " pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:51.418583 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.418566 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv6v" Sep 07 12:53:51.420554 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.420539 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtvwt\" (UniqueName: \"kubernetes.io/projected/7b8afbb2-9557-4821-8712-89cc16f472f4-kube-api-access-wtvwt\") pod \"network-check-target-m9c7v\" (UID: \"7b8afbb2-9557-4821-8712-89cc16f472f4\") " pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:51.582109 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.582081 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hpq9f"] Sep 07 12:53:51.585104 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:51.585080 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e766773_bcc2_4cb6_8df6_3dc08e9d59cc.slice/crio-352ebd5479a472a4358e25d04f49f32de6aa64b6dc13d6720ec26569f691ab4a WatchSource:0}: Error finding container 352ebd5479a472a4358e25d04f49f32de6aa64b6dc13d6720ec26569f691ab4a: Status 404 returned error can't find the container with id 352ebd5479a472a4358e25d04f49f32de6aa64b6dc13d6720ec26569f691ab4a Sep 07 12:53:51.696621 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.696598 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xfxxc\"" Sep 07 12:53:51.704903 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.704883 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:51.715773 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.715746 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pkv6v"] Sep 07 12:53:51.718992 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:51.718953 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda204f77_d893_4332_baab_4418cb7e72bc.slice/crio-ec0e07b76a213114048b30a1f9fb2699fd3ca289ea1a6ec2a3eee753c16620f3 WatchSource:0}: Error finding container ec0e07b76a213114048b30a1f9fb2699fd3ca289ea1a6ec2a3eee753c16620f3: Status 404 returned error can't find the container with id ec0e07b76a213114048b30a1f9fb2699fd3ca289ea1a6ec2a3eee753c16620f3 Sep 07 12:53:51.875558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.875524 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pkv6v" event={"ID":"da204f77-d893-4332-baab-4418cb7e72bc","Type":"ContainerStarted","Data":"ec0e07b76a213114048b30a1f9fb2699fd3ca289ea1a6ec2a3eee753c16620f3"} Sep 07 12:53:51.876550 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:51.876528 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hpq9f" event={"ID":"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc","Type":"ContainerStarted","Data":"352ebd5479a472a4358e25d04f49f32de6aa64b6dc13d6720ec26569f691ab4a"} Sep 07 12:53:52.002599 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:52.002510 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m9c7v"] Sep 07 12:53:52.004838 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:52.004819 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:52.004928 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:52.004865 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:53:52.005196 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:52.005181 2576 scope.go:117] "RemoveContainer" containerID="86922340083aeb2cb8765db6af4d3823a55cd2c2d7fce7a2a5bec795d46d6359" Sep 07 12:53:52.005364 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:53:52.005340 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-5fzst_openshift-console-operator(d4561d14-f779-4e6f-b63c-09f5d09bafed)\"" pod="openshift-console-operator/console-operator-85b454874b-5fzst" podUID="d4561d14-f779-4e6f-b63c-09f5d09bafed" Sep 07 12:53:52.006156 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:52.006139 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b8afbb2_9557_4821_8712_89cc16f472f4.slice/crio-f211d91b6e873d77f8ee26f3fc6b90228c7c45c933d9e4085ce575272aa74fa7 WatchSource:0}: Error finding container f211d91b6e873d77f8ee26f3fc6b90228c7c45c933d9e4085ce575272aa74fa7: Status 404 returned error can't find the container with id f211d91b6e873d77f8ee26f3fc6b90228c7c45c933d9e4085ce575272aa74fa7 Sep 07 12:53:52.881944 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:52.881910 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m9c7v" event={"ID":"7b8afbb2-9557-4821-8712-89cc16f472f4","Type":"ContainerStarted","Data":"a1845bb553450358f4153302c9901f49e614d0ca1e3ba806d46ddfaf19028668"} Sep 07 12:53:52.882347 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:52.881996 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m9c7v" event={"ID":"7b8afbb2-9557-4821-8712-89cc16f472f4","Type":"ContainerStarted","Data":"f211d91b6e873d77f8ee26f3fc6b90228c7c45c933d9e4085ce575272aa74fa7"} Sep 07 12:53:52.882347 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:52.882027 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:53:52.901507 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:52.901469 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m9c7v" podStartSLOduration=65.90144871 podStartE2EDuration="1m5.90144871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:52.90049593 +0000 UTC m=+66.931245704" watchObservedRunningTime="2026-09-07 12:53:52.90144871 +0000 UTC m=+66.932198527" Sep 07 12:53:53.636227 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.636189 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:53.636428 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.636389 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:53.636493 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.636427 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:53.637015 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.636962 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f54978ff-9e19-427c-bec3-602c1ed25a1e-service-ca-bundle\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:53.639521 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.639497 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f54978ff-9e19-427c-bec3-602c1ed25a1e-metrics-certs\") pod \"router-default-6998fdb454-4p474\" (UID: \"f54978ff-9e19-427c-bec3-602c1ed25a1e\") " pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:53.639635 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.639539 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls\") pod \"image-registry-c6bcbc855-x2htj\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:53.737088 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.737051 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:53.737245 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.737129 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jdjj8\" (UID: \"d5881837-ea29-48f0-865e-40542a004cf5\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:53.737245 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.737189 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:53.737245 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.737217 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-szxpg\" (UID: \"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:53.739816 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.739765 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5881837-ea29-48f0-865e-40542a004cf5-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jdjj8\" (UID: \"d5881837-ea29-48f0-865e-40542a004cf5\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:53.739944 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.739919 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5afdf94f-7a4f-483b-9f83-cab863ef2e7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-n6cnx\" (UID: \"5afdf94f-7a4f-483b-9f83-cab863ef2e7b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:53.740366 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.740346 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9fd689c9-28b9-4a27-a6a5-bf0133374ac8-metrics-tls\") pod \"dns-default-bq8pr\" (UID: \"9fd689c9-28b9-4a27-a6a5-bf0133374ac8\") " pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:53.741489 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.741471 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac7c410e-42b2-4bcd-9099-9b8d7244cc5e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-szxpg\" (UID: \"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:53.823762 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.823731 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v9hr7\"" Sep 07 12:53:53.832291 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.832258 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:53.894809 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.892193 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pkv6v" event={"ID":"da204f77-d893-4332-baab-4418cb7e72bc","Type":"ContainerStarted","Data":"c7fcc1e9aa2b6151c4929588a15f2e3f97e2894b23028872273c870242708c24"} Sep 07 12:53:53.894809 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.892250 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pkv6v" event={"ID":"da204f77-d893-4332-baab-4418cb7e72bc","Type":"ContainerStarted","Data":"804222bf418d3055eda8f271e41df4fea7530563d67570ed32b4ae8146b8f3e8"} Sep 07 12:53:53.904660 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.904631 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mns4n\"" Sep 07 12:53:53.904806 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.904785 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:53.919237 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.919171 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pkv6v" podStartSLOduration=65.803563085 podStartE2EDuration="1m6.919152351s" totalImagesPullingTime="1.115589266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:53.91909002 +0000 UTC m=+67.949839815" watchObservedRunningTime="2026-09-07 12:53:53.919152351 +0000 UTC m=+67.949902125" Sep 07 12:53:53.930138 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.930115 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-t55z9\"" Sep 07 12:53:53.938653 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.938625 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" Sep 07 12:53:53.951808 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.951787 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-v6g2k\"" Sep 07 12:53:53.961260 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.960867 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" Sep 07 12:53:53.986124 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.986097 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-txsb2\"" Sep 07 12:53:53.994627 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:53.994599 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" Sep 07 12:53:54.007856 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.006218 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8c6z8\"" Sep 07 12:53:54.016359 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.016282 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bq8pr" Sep 07 12:53:54.149424 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.149315 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c6bcbc855-x2htj"] Sep 07 12:53:54.152726 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:54.152696 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15ddd367_8b23_4e84_b27f_187d89b0a7bc.slice/crio-91f9a413c667262dc4b7e4ec2930544f89630c495ee9ef491f8978b70df6fc7e WatchSource:0}: Error finding container 91f9a413c667262dc4b7e4ec2930544f89630c495ee9ef491f8978b70df6fc7e: Status 404 returned error can't find the container with id 91f9a413c667262dc4b7e4ec2930544f89630c495ee9ef491f8978b70df6fc7e Sep 07 12:53:54.208515 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.208488 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6998fdb454-4p474"] Sep 07 12:53:54.210564 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:54.210535 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf54978ff_9e19_427c_bec3_602c1ed25a1e.slice/crio-129ef768a3e7ed6b3faeb1d42cf698c1592e52c04230fde3bde5a092a5ce4dc2 WatchSource:0}: Error finding container 129ef768a3e7ed6b3faeb1d42cf698c1592e52c04230fde3bde5a092a5ce4dc2: Status 404 returned error can't find the container with id 129ef768a3e7ed6b3faeb1d42cf698c1592e52c04230fde3bde5a092a5ce4dc2 Sep 07 12:53:54.220159 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.220131 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h"] Sep 07 12:53:54.223746 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.223725 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" Sep 07 12:53:54.225759 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.225735 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 07 12:53:54.225856 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.225767 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 07 12:53:54.226100 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.226080 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 07 12:53:54.226225 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.226101 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 07 12:53:54.233253 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.233226 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h"] Sep 07 12:53:54.236737 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.236714 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx"] Sep 07 12:53:54.261069 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.261043 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-jdjj8"] Sep 07 12:53:54.266667 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:54.266638 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5881837_ea29_48f0_865e_40542a004cf5.slice/crio-a884e767d6abc22ae17354fcade88cd96d4669d35444e3955665fca7aed28e2d WatchSource:0}: Error finding container a884e767d6abc22ae17354fcade88cd96d4669d35444e3955665fca7aed28e2d: Status 404 returned error can't find the container with id a884e767d6abc22ae17354fcade88cd96d4669d35444e3955665fca7aed28e2d Sep 07 12:53:54.288029 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.287928 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:54.301959 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.301884 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg"] Sep 07 12:53:54.331063 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.330995 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bq8pr"] Sep 07 12:53:54.342265 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.342185 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8520b6b8-22e8-4f04-9454-79f39748106c-tmp\") pod \"klusterlet-addon-workmgr-5788b5fd8f-2755h\" (UID: \"8520b6b8-22e8-4f04-9454-79f39748106c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" Sep 07 12:53:54.342414 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.342340 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8520b6b8-22e8-4f04-9454-79f39748106c-klusterlet-config\") pod \"klusterlet-addon-workmgr-5788b5fd8f-2755h\" (UID: \"8520b6b8-22e8-4f04-9454-79f39748106c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" Sep 07 12:53:54.342414 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.342367 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xnq8\" (UniqueName: \"kubernetes.io/projected/8520b6b8-22e8-4f04-9454-79f39748106c-kube-api-access-6xnq8\") pod \"klusterlet-addon-workmgr-5788b5fd8f-2755h\" (UID: \"8520b6b8-22e8-4f04-9454-79f39748106c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" Sep 07 12:53:54.444427 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.443459 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8520b6b8-22e8-4f04-9454-79f39748106c-klusterlet-config\") pod \"klusterlet-addon-workmgr-5788b5fd8f-2755h\" (UID: \"8520b6b8-22e8-4f04-9454-79f39748106c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" Sep 07 12:53:54.444427 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.443513 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xnq8\" (UniqueName: \"kubernetes.io/projected/8520b6b8-22e8-4f04-9454-79f39748106c-kube-api-access-6xnq8\") pod \"klusterlet-addon-workmgr-5788b5fd8f-2755h\" (UID: \"8520b6b8-22e8-4f04-9454-79f39748106c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" Sep 07 12:53:54.444427 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.443548 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8520b6b8-22e8-4f04-9454-79f39748106c-tmp\") pod \"klusterlet-addon-workmgr-5788b5fd8f-2755h\" (UID: \"8520b6b8-22e8-4f04-9454-79f39748106c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" Sep 07 12:53:54.444900 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.444851 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8520b6b8-22e8-4f04-9454-79f39748106c-tmp\") pod \"klusterlet-addon-workmgr-5788b5fd8f-2755h\" (UID: \"8520b6b8-22e8-4f04-9454-79f39748106c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" Sep 07 12:53:54.446942 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.446898 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8520b6b8-22e8-4f04-9454-79f39748106c-klusterlet-config\") pod \"klusterlet-addon-workmgr-5788b5fd8f-2755h\" (UID: \"8520b6b8-22e8-4f04-9454-79f39748106c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" Sep 07 12:53:54.453126 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.453103 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xnq8\" (UniqueName: \"kubernetes.io/projected/8520b6b8-22e8-4f04-9454-79f39748106c-kube-api-access-6xnq8\") pod \"klusterlet-addon-workmgr-5788b5fd8f-2755h\" (UID: \"8520b6b8-22e8-4f04-9454-79f39748106c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" Sep 07 12:53:54.464011 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.463986 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:54.545526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.545493 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" Sep 07 12:53:54.582380 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.582349 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:54.899026 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.898994 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6998fdb454-4p474" event={"ID":"f54978ff-9e19-427c-bec3-602c1ed25a1e","Type":"ContainerStarted","Data":"ee5d244f2b93dba8dd09ababf0a32692032e55711dcedfab44617b4d288eb4cd"} Sep 07 12:53:54.899521 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.899044 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6998fdb454-4p474" event={"ID":"f54978ff-9e19-427c-bec3-602c1ed25a1e","Type":"ContainerStarted","Data":"129ef768a3e7ed6b3faeb1d42cf698c1592e52c04230fde3bde5a092a5ce4dc2"} Sep 07 12:53:54.900929 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.900893 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" event={"ID":"15ddd367-8b23-4e84-b27f-187d89b0a7bc","Type":"ContainerStarted","Data":"0ecda0a5b872aa7efbef3a1253eb963c2d730e0e7328427a5715a26a126b73fb"} Sep 07 12:53:54.901084 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.900943 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" event={"ID":"15ddd367-8b23-4e84-b27f-187d89b0a7bc","Type":"ContainerStarted","Data":"91f9a413c667262dc4b7e4ec2930544f89630c495ee9ef491f8978b70df6fc7e"} Sep 07 12:53:54.901262 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.901239 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:53:54.902794 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.902752 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" event={"ID":"d5881837-ea29-48f0-865e-40542a004cf5","Type":"ContainerStarted","Data":"a884e767d6abc22ae17354fcade88cd96d4669d35444e3955665fca7aed28e2d"} Sep 07 12:53:54.904114 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.904093 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" event={"ID":"5afdf94f-7a4f-483b-9f83-cab863ef2e7b","Type":"ContainerStarted","Data":"4ada597008bfd9683d7ed64b7d97382619254beb61518f1ad497ed2e82e9ea66"} Sep 07 12:53:54.905524 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.905505 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:54.908478 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.908459 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:54.921197 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.921152 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6998fdb454-4p474" podStartSLOduration=40.92113558 podStartE2EDuration="40.92113558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:54.919620559 +0000 UTC m=+68.950370365" watchObservedRunningTime="2026-09-07 12:53:54.92113558 +0000 UTC m=+68.951885353" Sep 07 12:53:54.941798 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:54.941744 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" podStartSLOduration=67.941731131 podStartE2EDuration="1m7.941731131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:54.939823086 +0000 UTC m=+68.970573064" watchObservedRunningTime="2026-09-07 12:53:54.941731131 +0000 UTC m=+68.972480904" Sep 07 12:53:55.327454 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:55.327422 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:55.405829 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:55.405790 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:55.722244 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:55.722170 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd689c9_28b9_4a27_a6a5_bf0133374ac8.slice/crio-be0b61e079149116a7c61c9889456e2e5206cc309eb6c342932ce866235ace1c WatchSource:0}: Error finding container be0b61e079149116a7c61c9889456e2e5206cc309eb6c342932ce866235ace1c: Status 404 returned error can't find the container with id be0b61e079149116a7c61c9889456e2e5206cc309eb6c342932ce866235ace1c Sep 07 12:53:55.909598 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:55.909147 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hpq9f" event={"ID":"6e766773-bcc2-4cb6-8df6-3dc08e9d59cc","Type":"ContainerStarted","Data":"02315116837dbdf93cd07f1fc839ed28b3c5915be82daa8ca57557a44a244db5"} Sep 07 12:53:55.910593 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:55.910565 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bq8pr" event={"ID":"9fd689c9-28b9-4a27-a6a5-bf0133374ac8","Type":"ContainerStarted","Data":"be0b61e079149116a7c61c9889456e2e5206cc309eb6c342932ce866235ace1c"} Sep 07 12:53:55.913060 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:55.912914 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" event={"ID":"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e","Type":"ContainerStarted","Data":"21b3317281ce9f667c8713efaa85098c2fbd0d401cb8ec73cd0d889e0c62cfc5"} Sep 07 12:53:55.913060 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:55.913021 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:55.914878 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:55.914858 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6998fdb454-4p474" Sep 07 12:53:55.935357 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:55.935307 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hpq9f" podStartSLOduration=64.734216978 podStartE2EDuration="1m8.935289441s" totalImagesPullingTime="4.201072463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:55.930522081 +0000 UTC m=+69.961271853" watchObservedRunningTime="2026-09-07 12:53:55.935289441 +0000 UTC m=+69.966039212" Sep 07 12:53:55.999725 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:55.999463 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h"] Sep 07 12:53:56.004511 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:53:56.004478 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8520b6b8_22e8_4f04_9454_79f39748106c.slice/crio-59d44fcb3cd4a315fbb998731e92d780e3cafb8d1b7f7ac08a2a54fb858dc4d5 WatchSource:0}: Error finding container 59d44fcb3cd4a315fbb998731e92d780e3cafb8d1b7f7ac08a2a54fb858dc4d5: Status 404 returned error can't find the container with id 59d44fcb3cd4a315fbb998731e92d780e3cafb8d1b7f7ac08a2a54fb858dc4d5 Sep 07 12:53:56.920131 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:56.920031 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" event={"ID":"8520b6b8-22e8-4f04-9454-79f39748106c","Type":"ContainerStarted","Data":"59d44fcb3cd4a315fbb998731e92d780e3cafb8d1b7f7ac08a2a54fb858dc4d5"} Sep 07 12:53:57.139475 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:57.139439 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:57.218796 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:57.218711 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:59.383205 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:59.383172 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:59.486732 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:59.486698 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:59.573880 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:59.573838 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:59.645844 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:59.645777 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:53:59.937821 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:59.937751 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" event={"ID":"5afdf94f-7a4f-483b-9f83-cab863ef2e7b","Type":"ContainerStarted","Data":"6d888638c3249fa1d3de9a54b8394ad1f6f46e0684a13b40c3141df4ea142c61"} Sep 07 12:53:59.954109 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:53:59.954056 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-n6cnx" podStartSLOduration=41.206883552 podStartE2EDuration="45.953926597s" totalImagesPullingTime="4.747043045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:53:59.953627018 +0000 UTC m=+73.984376814" watchObservedRunningTime="2026-09-07 12:53:59.953926597 +0000 UTC m=+73.984676369" Sep 07 12:54:01.947806 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:01.947696 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" event={"ID":"d5881837-ea29-48f0-865e-40542a004cf5","Type":"ContainerStarted","Data":"e714deae2e4acb279f18611d49a3b19f1547908939b6764a983f35f1e64781fc"} Sep 07 12:54:01.949360 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:01.949326 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" event={"ID":"8520b6b8-22e8-4f04-9454-79f39748106c","Type":"ContainerStarted","Data":"d6e59083981ae9beb4f1653d7f6e7141bd5066601c28669080a688ca2ef43504"} Sep 07 12:54:01.949524 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:01.949462 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" Sep 07 12:54:01.951574 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:01.951548 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bq8pr" event={"ID":"9fd689c9-28b9-4a27-a6a5-bf0133374ac8","Type":"ContainerStarted","Data":"9aed7bb5ce95d9e421658277c2957a00fd881364d0ab8022451739e290045f36"} Sep 07 12:54:01.951697 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:01.951625 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" Sep 07 12:54:01.951697 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:01.951643 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bq8pr" event={"ID":"9fd689c9-28b9-4a27-a6a5-bf0133374ac8","Type":"ContainerStarted","Data":"b189dda1addf0a37c50521332ad3a78f1fa6903a8d88f1da1c1ad7e1ca06839b"} Sep 07 12:54:01.951697 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:01.951666 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bq8pr" Sep 07 12:54:01.953374 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:01.953351 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" event={"ID":"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e","Type":"ContainerStarted","Data":"e8a35ceda320262d2ec14ecb97992b7b857139653b8eb37706968880bbab5297"} Sep 07 12:54:01.953572 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:01.953389 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" event={"ID":"ac7c410e-42b2-4bcd-9099-9b8d7244cc5e","Type":"ContainerStarted","Data":"b3c6263a016981cec892ca8e422c6642d3d297cd4f774aca9d8294c052fad726"} Sep 07 12:54:01.964326 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:01.964263 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-jdjj8" podStartSLOduration=43.243026519 podStartE2EDuration="47.964248084s" totalImagesPullingTime="4.721221565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:54:01.964174135 +0000 UTC m=+75.994923905" watchObservedRunningTime="2026-09-07 12:54:01.964248084 +0000 UTC m=+75.994997855" Sep 07 12:54:01.981917 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:01.981860 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bq8pr" podStartSLOduration=35.535090521 podStartE2EDuration="40.981841229s" totalImagesPullingTime="5.446750708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:54:01.980838103 +0000 UTC m=+76.011587875" watchObservedRunningTime="2026-09-07 12:54:01.981841229 +0000 UTC m=+76.012590999" Sep 07 12:54:01.996849 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:01.996786 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5788b5fd8f-2755h" podStartSLOduration=2.448955543 podStartE2EDuration="7.996762664s" totalImagesPullingTime="5.547807121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:54:01.995991041 +0000 UTC m=+76.026740828" watchObservedRunningTime="2026-09-07 12:54:01.996762664 +0000 UTC m=+76.027512436" Sep 07 12:54:02.016504 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:02.016455 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-szxpg" podStartSLOduration=44.798949256 podStartE2EDuration="48.016438204s" totalImagesPullingTime="3.217488948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:54:02.014988139 +0000 UTC m=+76.045737923" watchObservedRunningTime="2026-09-07 12:54:02.016438204 +0000 UTC m=+76.047187975" Sep 07 12:54:02.135379 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:02.135344 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:02.232931 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:02.232818 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:02.476423 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:02.476388 2576 scope.go:117] "RemoveContainer" containerID="86922340083aeb2cb8765db6af4d3823a55cd2c2d7fce7a2a5bec795d46d6359" Sep 07 12:54:02.476629 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:54:02.476605 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-5fzst_openshift-console-operator(d4561d14-f779-4e6f-b63c-09f5d09bafed)\"" pod="openshift-console-operator/console-operator-85b454874b-5fzst" podUID="d4561d14-f779-4e6f-b63c-09f5d09bafed" Sep 07 12:54:02.769398 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:02.769355 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:02.870313 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:02.870282 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:03.607579 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:03.607535 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:04.477406 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:04.477362 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:05.328496 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:05.328454 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:05.400987 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:05.400937 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:06.358257 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:06.358222 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:09.081066 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.080929 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7pj2l"] Sep 07 12:54:09.085592 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.085576 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.095055 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.095037 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 07 12:54:09.095415 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.095400 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l5rwz\"" Sep 07 12:54:09.095519 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.095407 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 07 12:54:09.095519 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.095433 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 07 12:54:09.098133 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.098114 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 07 12:54:09.166089 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.166060 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-wtmp\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.166089 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.166089 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctd5v\" (UniqueName: \"kubernetes.io/projected/4b424863-bc8e-4e8a-9965-184185aa3dcf-kube-api-access-ctd5v\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.166324 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.166116 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.166324 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.166133 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b424863-bc8e-4e8a-9965-184185aa3dcf-metrics-client-ca\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.166324 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.166225 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-textfile\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.166324 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.166276 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b424863-bc8e-4e8a-9965-184185aa3dcf-sys\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.166324 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.166315 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-accelerators-collector-config\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.166520 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.166344 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-tls\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.166520 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.166370 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b424863-bc8e-4e8a-9965-184185aa3dcf-root\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.266889 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.266859 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-wtmp\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267051 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.266895 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctd5v\" (UniqueName: \"kubernetes.io/projected/4b424863-bc8e-4e8a-9965-184185aa3dcf-kube-api-access-ctd5v\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267051 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.266919 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267051 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.266942 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b424863-bc8e-4e8a-9965-184185aa3dcf-metrics-client-ca\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267051 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.267000 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-textfile\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267051 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.267035 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b424863-bc8e-4e8a-9965-184185aa3dcf-sys\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267297 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.267061 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-accelerators-collector-config\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267297 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.267067 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-wtmp\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267297 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.267088 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-tls\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267297 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.267106 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b424863-bc8e-4e8a-9965-184185aa3dcf-sys\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267297 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.267112 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b424863-bc8e-4e8a-9965-184185aa3dcf-root\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267297 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.267143 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b424863-bc8e-4e8a-9965-184185aa3dcf-root\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267297 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:54:09.267237 2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 07 12:54:09.267570 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:54:09.267333 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-tls podName:4b424863-bc8e-4e8a-9965-184185aa3dcf nodeName:}" failed. No retries permitted until 2026-09-07 12:54:09.767311004 +0000 UTC m=+83.798060752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-tls") pod "node-exporter-7pj2l" (UID: "4b424863-bc8e-4e8a-9965-184185aa3dcf") : secret "node-exporter-tls" not found Sep 07 12:54:09.267570 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.267402 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-textfile\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267781 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.267759 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-accelerators-collector-config\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.267829 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.267763 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b424863-bc8e-4e8a-9965-184185aa3dcf-metrics-client-ca\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.269538 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.269520 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.276271 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.276254 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctd5v\" (UniqueName: \"kubernetes.io/projected/4b424863-bc8e-4e8a-9965-184185aa3dcf-kube-api-access-ctd5v\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.608373 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.608331 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:09.710690 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.710649 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:09.773117 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.773084 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-tls\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.775629 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.775606 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b424863-bc8e-4e8a-9965-184185aa3dcf-node-exporter-tls\") pod \"node-exporter-7pj2l\" (UID: \"4b424863-bc8e-4e8a-9965-184185aa3dcf\") " pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:09.994873 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:09.994803 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7pj2l" Sep 07 12:54:10.007574 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:54:10.007540 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b424863_bc8e_4e8a_9965_184185aa3dcf.slice/crio-c9e0b4b5493d8c151416de2266d30d27d9e231409f1793504875c4ec522ef45e WatchSource:0}: Error finding container c9e0b4b5493d8c151416de2266d30d27d9e231409f1793504875c4ec522ef45e: Status 404 returned error can't find the container with id c9e0b4b5493d8c151416de2266d30d27d9e231409f1793504875c4ec522ef45e Sep 07 12:54:10.049875 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:10.049829 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:10.156505 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:10.156465 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:10.454283 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:10.454251 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:10.994770 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:10.994739 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7pj2l" event={"ID":"4b424863-bc8e-4e8a-9965-184185aa3dcf","Type":"ContainerStarted","Data":"c9e0b4b5493d8c151416de2266d30d27d9e231409f1793504875c4ec522ef45e"} Sep 07 12:54:11.959105 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:11.959075 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bq8pr" Sep 07 12:54:12.000951 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:12.000921 2576 generic.go:356] "Generic (PLEG): container finished" podID="4b424863-bc8e-4e8a-9965-184185aa3dcf" containerID="4028cc69719f21541d08bb74b08690aeda1c5d8f51893c5a2c8907373677cb4f" exitCode=0 Sep 07 12:54:12.001160 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:12.001044 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7pj2l" event={"ID":"4b424863-bc8e-4e8a-9965-184185aa3dcf","Type":"ContainerDied","Data":"4028cc69719f21541d08bb74b08690aeda1c5d8f51893c5a2c8907373677cb4f"} Sep 07 12:54:12.007585 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:12.007554 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:12.109036 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:12.109011 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:12.212838 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:12.212748 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:12.410523 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:12.410486 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:12.940989 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:12.940930 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:13.011104 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:13.011058 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7pj2l" event={"ID":"4b424863-bc8e-4e8a-9965-184185aa3dcf","Type":"ContainerStarted","Data":"8565363a9d627765f31085a03d1720c465daeabaa50220432bcfe457d9f28c26"} Sep 07 12:54:13.011558 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:13.011153 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7pj2l" event={"ID":"4b424863-bc8e-4e8a-9965-184185aa3dcf","Type":"ContainerStarted","Data":"0757306bba28ba6533c95845ec577b87be3c2d6d980a50a9cab64a6503fd8c58"} Sep 07 12:54:13.014680 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:13.014641 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:13.030868 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:13.030824 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7pj2l" podStartSLOduration=2.831208641 podStartE2EDuration="4.030811346s" totalImagesPullingTime="1.130346987s" totalInitContainerRuntime="69.255718ms" isStatefulPod=true podCreationTimestamp="2026-09-07 12:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:54:13.029873323 +0000 UTC m=+87.060623119" watchObservedRunningTime="2026-09-07 12:54:13.030811346 +0000 UTC m=+87.061561116" Sep 07 12:54:13.692578 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:13.692538 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:13.791822 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:13.791791 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:14.610185 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:14.610145 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:14.678163 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:14.678136 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:15.306528 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:15.306495 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:15.476872 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:15.476845 2576 scope.go:117] "RemoveContainer" containerID="86922340083aeb2cb8765db6af4d3823a55cd2c2d7fce7a2a5bec795d46d6359" Sep 07 12:54:15.917735 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:15.917709 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:54:16.025800 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:16.025776 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 12:54:16.025961 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:16.025878 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-5fzst" event={"ID":"d4561d14-f779-4e6f-b63c-09f5d09bafed","Type":"ContainerStarted","Data":"50f0ea4e3902d5b3631b1d69aebf55e3bd4b406a81f218d6cc0ac0522f2ecda3"} Sep 07 12:54:16.026190 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:16.026174 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:54:16.046800 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:16.046747 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-5fzst" podStartSLOduration=55.922164087 podStartE2EDuration="1m2.046729343s" totalImagesPullingTime="6.124565256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:54:16.045271173 +0000 UTC m=+90.076020943" watchObservedRunningTime="2026-09-07 12:54:16.046729343 +0000 UTC m=+90.077479118" Sep 07 12:54:16.137936 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:16.137907 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c6bcbc855-x2htj"] Sep 07 12:54:16.176740 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:16.176643 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:16.253753 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:16.253720 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:16.429798 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:16.429724 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-5fzst" Sep 07 12:54:16.597183 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:16.597147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:16.673831 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:16.673779 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:16.753788 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:16.753690 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:16.933938 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:16.933901 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:17.298467 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:17.298432 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:17.366385 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:17.366352 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:17.698117 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:17.698025 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:17.763243 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:17.763209 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:18.123357 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:18.123320 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:18.314111 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:18.314079 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:18.722115 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:18.722075 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:18.904780 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:18.904749 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:19.007929 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:19.007850 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:19.139102 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:19.139068 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:19.240609 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:19.240562 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:22.934430 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:22.934396 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:23.000244 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:23.000211 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:23.894789 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:23.894757 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m9c7v" Sep 07 12:54:25.160962 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:25.160921 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:25.219391 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:25.219305 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:25.534262 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:25.534230 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:25.595364 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:25.595334 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:27.630891 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:27.630856 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:27.728927 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:27.728880 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:27.894814 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:27.894736 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:27.977757 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:27.977724 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:31.930861 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:31.930821 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:34.377673 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:34.377639 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:34.477748 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:34.477701 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:36.143610 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:36.143575 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:36.217687 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:36.217644 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:39.938372 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:39.938332 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:41.165115 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.165055 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" podUID="15ddd367-8b23-4e84-b27f-187d89b0a7bc" containerName="registry" containerID="cri-o://0ecda0a5b872aa7efbef3a1253eb963c2d730e0e7328427a5715a26a126b73fb" gracePeriod=30 Sep 07 12:54:41.399203 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.399180 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:54:41.418722 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.418649 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15ddd367-8b23-4e84-b27f-187d89b0a7bc-trusted-ca\") pod \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " Sep 07 12:54:41.418722 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.418707 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15ddd367-8b23-4e84-b27f-187d89b0a7bc-ca-trust-extracted\") pod \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " Sep 07 12:54:41.418898 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.418804 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-certificates\") pod \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " Sep 07 12:54:41.418898 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.418859 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls\") pod \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " Sep 07 12:54:41.419036 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.418896 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-bound-sa-token\") pod \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " Sep 07 12:54:41.419036 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.418953 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15ddd367-8b23-4e84-b27f-187d89b0a7bc-installation-pull-secrets\") pod \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " Sep 07 12:54:41.419036 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.419002 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15ddd367-8b23-4e84-b27f-187d89b0a7bc-image-registry-private-configuration\") pod \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " Sep 07 12:54:41.419715 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.419222 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndw9l\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-kube-api-access-ndw9l\") pod \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\" (UID: \"15ddd367-8b23-4e84-b27f-187d89b0a7bc\") " Sep 07 12:54:41.419715 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.419281 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "15ddd367-8b23-4e84-b27f-187d89b0a7bc" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 12:54:41.419715 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.419464 2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-certificates\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 12:54:41.422151 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.422094 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15ddd367-8b23-4e84-b27f-187d89b0a7bc-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "15ddd367-8b23-4e84-b27f-187d89b0a7bc" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 12:54:41.423660 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.423633 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-kube-api-access-ndw9l" (OuterVolumeSpecName: "kube-api-access-ndw9l") pod "15ddd367-8b23-4e84-b27f-187d89b0a7bc" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc"). InnerVolumeSpecName "kube-api-access-ndw9l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 12:54:41.423779 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.423654 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15ddd367-8b23-4e84-b27f-187d89b0a7bc-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "15ddd367-8b23-4e84-b27f-187d89b0a7bc" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 12:54:41.423917 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.423890 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15ddd367-8b23-4e84-b27f-187d89b0a7bc-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "15ddd367-8b23-4e84-b27f-187d89b0a7bc" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 12:54:41.424050 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.423931 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "15ddd367-8b23-4e84-b27f-187d89b0a7bc" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 12:54:41.424308 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.423822 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "15ddd367-8b23-4e84-b27f-187d89b0a7bc" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 12:54:41.434573 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.434545 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15ddd367-8b23-4e84-b27f-187d89b0a7bc-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "15ddd367-8b23-4e84-b27f-187d89b0a7bc" (UID: "15ddd367-8b23-4e84-b27f-187d89b0a7bc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 12:54:41.519777 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.519758 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-registry-tls\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 12:54:41.519777 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.519776 2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-bound-sa-token\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 12:54:41.519912 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.519785 2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15ddd367-8b23-4e84-b27f-187d89b0a7bc-installation-pull-secrets\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 12:54:41.519912 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.519793 2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15ddd367-8b23-4e84-b27f-187d89b0a7bc-image-registry-private-configuration\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 12:54:41.519912 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.519803 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndw9l\" (UniqueName: \"kubernetes.io/projected/15ddd367-8b23-4e84-b27f-187d89b0a7bc-kube-api-access-ndw9l\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 12:54:41.519912 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.519810 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15ddd367-8b23-4e84-b27f-187d89b0a7bc-trusted-ca\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 12:54:41.519912 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.519818 2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15ddd367-8b23-4e84-b27f-187d89b0a7bc-ca-trust-extracted\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 12:54:41.844680 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.844625 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:41.901320 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:41.901292 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:42.121335 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:42.121258 2576 generic.go:356] "Generic (PLEG): container finished" podID="15ddd367-8b23-4e84-b27f-187d89b0a7bc" containerID="0ecda0a5b872aa7efbef3a1253eb963c2d730e0e7328427a5715a26a126b73fb" exitCode=0 Sep 07 12:54:42.121335 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:42.121294 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" event={"ID":"15ddd367-8b23-4e84-b27f-187d89b0a7bc","Type":"ContainerDied","Data":"0ecda0a5b872aa7efbef3a1253eb963c2d730e0e7328427a5715a26a126b73fb"} Sep 07 12:54:42.121335 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:42.121306 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" Sep 07 12:54:42.121560 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:42.121339 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c6bcbc855-x2htj" event={"ID":"15ddd367-8b23-4e84-b27f-187d89b0a7bc","Type":"ContainerDied","Data":"91f9a413c667262dc4b7e4ec2930544f89630c495ee9ef491f8978b70df6fc7e"} Sep 07 12:54:42.121560 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:42.121359 2576 scope.go:117] "RemoveContainer" containerID="0ecda0a5b872aa7efbef3a1253eb963c2d730e0e7328427a5715a26a126b73fb" Sep 07 12:54:42.132896 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:42.132875 2576 scope.go:117] "RemoveContainer" containerID="0ecda0a5b872aa7efbef3a1253eb963c2d730e0e7328427a5715a26a126b73fb" Sep 07 12:54:42.133208 ip-10-0-142-1 kubenswrapper[2576]: E0907 12:54:42.133186 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ecda0a5b872aa7efbef3a1253eb963c2d730e0e7328427a5715a26a126b73fb\": container with ID starting with 0ecda0a5b872aa7efbef3a1253eb963c2d730e0e7328427a5715a26a126b73fb not found: ID does not exist" containerID="0ecda0a5b872aa7efbef3a1253eb963c2d730e0e7328427a5715a26a126b73fb" Sep 07 12:54:42.133287 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:42.133220 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ecda0a5b872aa7efbef3a1253eb963c2d730e0e7328427a5715a26a126b73fb"} err="failed to get container status \"0ecda0a5b872aa7efbef3a1253eb963c2d730e0e7328427a5715a26a126b73fb\": rpc error: code = NotFound desc = could not find container \"0ecda0a5b872aa7efbef3a1253eb963c2d730e0e7328427a5715a26a126b73fb\": container with ID starting with 0ecda0a5b872aa7efbef3a1253eb963c2d730e0e7328427a5715a26a126b73fb not found: ID does not exist" Sep 07 12:54:42.147312 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:42.147280 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c6bcbc855-x2htj"] Sep 07 12:54:42.152380 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:42.152356 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-c6bcbc855-x2htj"] Sep 07 12:54:42.481655 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:42.481563 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15ddd367-8b23-4e84-b27f-187d89b0a7bc" path="/var/lib/kubelet/pods/15ddd367-8b23-4e84-b27f-187d89b0a7bc/volumes" Sep 07 12:54:48.543460 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:48.543411 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:48.608175 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:48.608142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:49.341059 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:49.341026 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:49.440915 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:49.440878 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:50.533780 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:50.533742 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:50.605420 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:50.605386 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:54.166386 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:54.166299 2576 generic.go:356] "Generic (PLEG): container finished" podID="c73cfc77-cf60-4c83-9d2c-1f5a56779218" containerID="eef58704faa3ff73b34113c7960898b3e9993b1a868324792141d43fc73ecd8a" exitCode=0 Sep 07 12:54:54.166386 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:54.166349 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" event={"ID":"c73cfc77-cf60-4c83-9d2c-1f5a56779218","Type":"ContainerDied","Data":"eef58704faa3ff73b34113c7960898b3e9993b1a868324792141d43fc73ecd8a"} Sep 07 12:54:54.166814 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:54.166693 2576 scope.go:117] "RemoveContainer" containerID="eef58704faa3ff73b34113c7960898b3e9993b1a868324792141d43fc73ecd8a" Sep 07 12:54:55.172373 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:55.172335 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-9r4xn" event={"ID":"c73cfc77-cf60-4c83-9d2c-1f5a56779218","Type":"ContainerStarted","Data":"4b5090215d797f8fb3628c083c9b7c293ab8f9811c025546a4b24ec7cfca849f"} Sep 07 12:54:55.173802 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:55.173780 2576 generic.go:356] "Generic (PLEG): container finished" podID="7b88887b-d35e-4197-987a-7c91b196cba8" containerID="196f04fe63838f49709b05925fe8036a18e747626568f34dfe5df26ab1f80927" exitCode=0 Sep 07 12:54:55.173893 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:55.173824 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" event={"ID":"7b88887b-d35e-4197-987a-7c91b196cba8","Type":"ContainerDied","Data":"196f04fe63838f49709b05925fe8036a18e747626568f34dfe5df26ab1f80927"} Sep 07 12:54:55.174124 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:55.174112 2576 scope.go:117] "RemoveContainer" containerID="196f04fe63838f49709b05925fe8036a18e747626568f34dfe5df26ab1f80927" Sep 07 12:54:56.179926 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:56.179888 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-trlrt" event={"ID":"7b88887b-d35e-4197-987a-7c91b196cba8","Type":"ContainerStarted","Data":"1b00116793538a6f899a9e8015558b1662be3d2a01c54c7a1ca67540c03d3813"} Sep 07 12:54:58.502546 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:58.502501 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:58.602839 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:58.602802 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:54:58.671198 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:54:58.671154 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:00.433616 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:00.433581 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:04.361493 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:04.361442 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:04.443610 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:04.443574 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:05.212278 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:05.212246 2576 generic.go:356] "Generic (PLEG): container finished" podID="c18ca5b5-32d3-4953-a134-397fce347a66" containerID="dcc44219299f3655416b63b9f67146b9b8b37b83c2cf21c756f90e80cdbb3054" exitCode=0 Sep 07 12:55:05.212451 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:05.212329 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" event={"ID":"c18ca5b5-32d3-4953-a134-397fce347a66","Type":"ContainerDied","Data":"dcc44219299f3655416b63b9f67146b9b8b37b83c2cf21c756f90e80cdbb3054"} Sep 07 12:55:05.212691 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:05.212678 2576 scope.go:117] "RemoveContainer" containerID="dcc44219299f3655416b63b9f67146b9b8b37b83c2cf21c756f90e80cdbb3054" Sep 07 12:55:05.896559 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:05.896523 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:06.002645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:06.002603 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:06.218102 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:06.218014 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-8nr54" event={"ID":"c18ca5b5-32d3-4953-a134-397fce347a66","Type":"ContainerStarted","Data":"0dbf0ebeef7f8b238ee27c2338f5bd8bc9b9753c0d12bfda8122953c4e9ae771"} Sep 07 12:55:22.959243 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:22.959207 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:28.482268 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:28.482235 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:28.835965 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:28.835936 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:37.404885 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:37.404849 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:37.471437 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:37.471400 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:37.653434 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:37.653351 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:37.757688 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:37.757593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:38.152407 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:38.152360 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:38.211797 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:38.211759 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:48.728378 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:48.728341 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:48.827706 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:48.827672 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:55:48.927982 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:55:48.927928 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:56:24.758746 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:56:24.758698 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:56:36.069122 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:56:36.069079 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:56:36.184899 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:56:36.184848 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:56:53.756335 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:56:53.756296 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:56:53.823338 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:56:53.823301 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:57:02.385585 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:02.385551 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:57:02.480283 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:02.480220 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:57:05.769406 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:05.769357 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:57:05.861817 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:05.861776 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:57:12.998626 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:12.998521 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:57:13.111626 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:13.111594 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:57:13.202911 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:13.202874 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:57:19.346629 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:19.346580 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:57:19.450893 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:19.450854 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:57:19.517911 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:19.517876 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:57:46.383998 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:46.383956 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 12:57:46.389129 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:46.389098 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 12:57:46.394082 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:46.394059 2576 kubelet.go:1648] "Image garbage collection succeeded" Sep 07 12:57:56.182810 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:56.182771 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:57:57.670615 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:57:57.670580 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:18.705775 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:18.705729 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:19.693900 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:19.693868 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:22.523457 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:22.523413 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:22.765876 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:22.765845 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:23.507485 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:23.507452 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:23.621526 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:23.621494 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:24.345461 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.345426 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx"] Sep 07 12:58:24.345740 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.345728 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15ddd367-8b23-4e84-b27f-187d89b0a7bc" containerName="registry" Sep 07 12:58:24.345783 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.345742 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="15ddd367-8b23-4e84-b27f-187d89b0a7bc" containerName="registry" Sep 07 12:58:24.345812 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.345801 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="15ddd367-8b23-4e84-b27f-187d89b0a7bc" containerName="registry" Sep 07 12:58:24.348797 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.348780 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" Sep 07 12:58:24.350636 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.350612 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 07 12:58:24.350877 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.350863 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 07 12:58:24.350934 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.350873 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xfcv5\"" Sep 07 12:58:24.357839 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.357815 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx"] Sep 07 12:58:24.484622 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.484592 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ed04487-c991-42ff-a884-1916cae362ba-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx\" (UID: \"7ed04487-c991-42ff-a884-1916cae362ba\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" Sep 07 12:58:24.484783 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.484636 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ed04487-c991-42ff-a884-1916cae362ba-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx\" (UID: \"7ed04487-c991-42ff-a884-1916cae362ba\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" Sep 07 12:58:24.484783 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.484667 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6rdd\" (UniqueName: \"kubernetes.io/projected/7ed04487-c991-42ff-a884-1916cae362ba-kube-api-access-h6rdd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx\" (UID: \"7ed04487-c991-42ff-a884-1916cae362ba\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" Sep 07 12:58:24.585303 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.585260 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ed04487-c991-42ff-a884-1916cae362ba-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx\" (UID: \"7ed04487-c991-42ff-a884-1916cae362ba\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" Sep 07 12:58:24.585303 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.585317 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ed04487-c991-42ff-a884-1916cae362ba-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx\" (UID: \"7ed04487-c991-42ff-a884-1916cae362ba\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" Sep 07 12:58:24.585551 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.585348 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6rdd\" (UniqueName: \"kubernetes.io/projected/7ed04487-c991-42ff-a884-1916cae362ba-kube-api-access-h6rdd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx\" (UID: \"7ed04487-c991-42ff-a884-1916cae362ba\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" Sep 07 12:58:24.585639 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.585619 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ed04487-c991-42ff-a884-1916cae362ba-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx\" (UID: \"7ed04487-c991-42ff-a884-1916cae362ba\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" Sep 07 12:58:24.585724 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.585704 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ed04487-c991-42ff-a884-1916cae362ba-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx\" (UID: \"7ed04487-c991-42ff-a884-1916cae362ba\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" Sep 07 12:58:24.596680 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.596619 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6rdd\" (UniqueName: \"kubernetes.io/projected/7ed04487-c991-42ff-a884-1916cae362ba-kube-api-access-h6rdd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx\" (UID: \"7ed04487-c991-42ff-a884-1916cae362ba\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" Sep 07 12:58:24.657392 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.657370 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" Sep 07 12:58:24.749464 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.749432 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:24.847215 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.847124 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:24.927650 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.927622 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx"] Sep 07 12:58:24.930520 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.930494 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9qqb5"] Sep 07 12:58:24.931048 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:58:24.931023 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ed04487_c991_42ff_a884_1916cae362ba.slice/crio-7add6a027c5856011bf953250749f26515a9b9f38e620f7d084be5ba15133b98 WatchSource:0}: Error finding container 7add6a027c5856011bf953250749f26515a9b9f38e620f7d084be5ba15133b98: Status 404 returned error can't find the container with id 7add6a027c5856011bf953250749f26515a9b9f38e620f7d084be5ba15133b98 Sep 07 12:58:24.932885 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.932869 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 12:58:24.934729 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.934712 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9qqb5" Sep 07 12:58:24.936646 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.936627 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 07 12:58:24.937200 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.937181 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 07 12:58:24.937280 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.937182 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-xfpcd\"" Sep 07 12:58:24.939237 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.939215 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" event={"ID":"7ed04487-c991-42ff-a884-1916cae362ba","Type":"ContainerStarted","Data":"7add6a027c5856011bf953250749f26515a9b9f38e620f7d084be5ba15133b98"} Sep 07 12:58:24.950777 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.950758 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9qqb5"] Sep 07 12:58:24.988846 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.988822 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b310be3-513d-4682-880f-288c17f705a6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9qqb5\" (UID: \"1b310be3-513d-4682-880f-288c17f705a6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9qqb5" Sep 07 12:58:24.988954 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:24.988855 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pn82\" (UniqueName: \"kubernetes.io/projected/1b310be3-513d-4682-880f-288c17f705a6-kube-api-access-8pn82\") pod \"cert-manager-webhook-6486ffd997-9qqb5\" (UID: \"1b310be3-513d-4682-880f-288c17f705a6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9qqb5" Sep 07 12:58:25.090192 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:25.090162 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b310be3-513d-4682-880f-288c17f705a6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9qqb5\" (UID: \"1b310be3-513d-4682-880f-288c17f705a6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9qqb5" Sep 07 12:58:25.090192 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:25.090196 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pn82\" (UniqueName: \"kubernetes.io/projected/1b310be3-513d-4682-880f-288c17f705a6-kube-api-access-8pn82\") pod \"cert-manager-webhook-6486ffd997-9qqb5\" (UID: \"1b310be3-513d-4682-880f-288c17f705a6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9qqb5" Sep 07 12:58:25.101232 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:25.101163 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b310be3-513d-4682-880f-288c17f705a6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9qqb5\" (UID: \"1b310be3-513d-4682-880f-288c17f705a6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9qqb5" Sep 07 12:58:25.101328 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:25.101248 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pn82\" (UniqueName: \"kubernetes.io/projected/1b310be3-513d-4682-880f-288c17f705a6-kube-api-access-8pn82\") pod \"cert-manager-webhook-6486ffd997-9qqb5\" (UID: \"1b310be3-513d-4682-880f-288c17f705a6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9qqb5" Sep 07 12:58:25.252571 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:25.252530 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9qqb5" Sep 07 12:58:25.524670 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:25.524605 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9qqb5"] Sep 07 12:58:25.527339 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:58:25.527313 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b310be3_513d_4682_880f_288c17f705a6.slice/crio-6ffd31bc26e4d576298cf5ddecfe0b04616b3411c78333c261927b08426e81e8 WatchSource:0}: Error finding container 6ffd31bc26e4d576298cf5ddecfe0b04616b3411c78333c261927b08426e81e8: Status 404 returned error can't find the container with id 6ffd31bc26e4d576298cf5ddecfe0b04616b3411c78333c261927b08426e81e8 Sep 07 12:58:25.944889 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:25.944845 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9qqb5" event={"ID":"1b310be3-513d-4682-880f-288c17f705a6","Type":"ContainerStarted","Data":"6ffd31bc26e4d576298cf5ddecfe0b04616b3411c78333c261927b08426e81e8"} Sep 07 12:58:26.814488 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:26.814427 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:27.201248 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:27.201168 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:29.121543 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:29.121500 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:29.189286 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:29.189243 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:30.969068 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:30.969033 2576 generic.go:356] "Generic (PLEG): container finished" podID="7ed04487-c991-42ff-a884-1916cae362ba" containerID="9aa680281588a074187c31fed8ef7d08bcd7428edb37a91d10b217b69ee1b1c5" exitCode=0 Sep 07 12:58:30.969454 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:30.969125 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" event={"ID":"7ed04487-c991-42ff-a884-1916cae362ba","Type":"ContainerDied","Data":"9aa680281588a074187c31fed8ef7d08bcd7428edb37a91d10b217b69ee1b1c5"} Sep 07 12:58:34.956623 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:34.956514 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:34.988589 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:34.988552 2576 generic.go:356] "Generic (PLEG): container finished" podID="7ed04487-c991-42ff-a884-1916cae362ba" containerID="e5ade4170fc4bdbbce33e096a8abfb5d72038451cf2e9bb6da1bb317f49c1444" exitCode=0 Sep 07 12:58:34.988729 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:34.988650 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" event={"ID":"7ed04487-c991-42ff-a884-1916cae362ba","Type":"ContainerDied","Data":"e5ade4170fc4bdbbce33e096a8abfb5d72038451cf2e9bb6da1bb317f49c1444"} Sep 07 12:58:34.990321 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:34.990300 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9qqb5" event={"ID":"1b310be3-513d-4682-880f-288c17f705a6","Type":"ContainerStarted","Data":"ed2fdc3f422b2da26ba5e9570fdf49bde38e1433da1e905292f39560075dfdf1"} Sep 07 12:58:34.990424 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:34.990405 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9qqb5" Sep 07 12:58:35.021187 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:35.021144 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:35.023257 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:35.023215 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-9qqb5" podStartSLOduration=1.872878703 podStartE2EDuration="11.023201201s" totalImagesPullingTime="9.150322498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:58:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:58:35.022460688 +0000 UTC m=+349.053210470" watchObservedRunningTime="2026-09-07 12:58:35.023201201 +0000 UTC m=+349.053950972" Sep 07 12:58:35.379412 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:35.379372 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:40.997393 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:40.997311 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9qqb5" Sep 07 12:58:42.025276 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:42.025246 2576 generic.go:356] "Generic (PLEG): container finished" podID="7ed04487-c991-42ff-a884-1916cae362ba" containerID="2f5d58ee5a1d9d1b9937fd18a863ab64c1cfda9bd1985c96f1ca781e7d95ce71" exitCode=0 Sep 07 12:58:42.025643 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:42.025309 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" event={"ID":"7ed04487-c991-42ff-a884-1916cae362ba","Type":"ContainerDied","Data":"2f5d58ee5a1d9d1b9937fd18a863ab64c1cfda9bd1985c96f1ca781e7d95ce71"} Sep 07 12:58:43.155116 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:43.155090 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" Sep 07 12:58:43.252442 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:43.252412 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ed04487-c991-42ff-a884-1916cae362ba-util\") pod \"7ed04487-c991-42ff-a884-1916cae362ba\" (UID: \"7ed04487-c991-42ff-a884-1916cae362ba\") " Sep 07 12:58:43.252590 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:43.252455 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6rdd\" (UniqueName: \"kubernetes.io/projected/7ed04487-c991-42ff-a884-1916cae362ba-kube-api-access-h6rdd\") pod \"7ed04487-c991-42ff-a884-1916cae362ba\" (UID: \"7ed04487-c991-42ff-a884-1916cae362ba\") " Sep 07 12:58:43.252590 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:43.252481 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ed04487-c991-42ff-a884-1916cae362ba-bundle\") pod \"7ed04487-c991-42ff-a884-1916cae362ba\" (UID: \"7ed04487-c991-42ff-a884-1916cae362ba\") " Sep 07 12:58:43.252866 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:43.252841 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ed04487-c991-42ff-a884-1916cae362ba-bundle" (OuterVolumeSpecName: "bundle") pod "7ed04487-c991-42ff-a884-1916cae362ba" (UID: "7ed04487-c991-42ff-a884-1916cae362ba"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 12:58:43.254802 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:43.254783 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ed04487-c991-42ff-a884-1916cae362ba-kube-api-access-h6rdd" (OuterVolumeSpecName: "kube-api-access-h6rdd") pod "7ed04487-c991-42ff-a884-1916cae362ba" (UID: "7ed04487-c991-42ff-a884-1916cae362ba"). InnerVolumeSpecName "kube-api-access-h6rdd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 12:58:43.256769 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:43.256750 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ed04487-c991-42ff-a884-1916cae362ba-util" (OuterVolumeSpecName: "util") pod "7ed04487-c991-42ff-a884-1916cae362ba" (UID: "7ed04487-c991-42ff-a884-1916cae362ba"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 12:58:43.353232 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:43.353153 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ed04487-c991-42ff-a884-1916cae362ba-util\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 12:58:43.353232 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:43.353183 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6rdd\" (UniqueName: \"kubernetes.io/projected/7ed04487-c991-42ff-a884-1916cae362ba-kube-api-access-h6rdd\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 12:58:43.353232 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:43.353197 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ed04487-c991-42ff-a884-1916cae362ba-bundle\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 12:58:44.035363 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:44.035338 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" Sep 07 12:58:44.035363 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:44.035343 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hjpx" event={"ID":"7ed04487-c991-42ff-a884-1916cae362ba","Type":"ContainerDied","Data":"7add6a027c5856011bf953250749f26515a9b9f38e620f7d084be5ba15133b98"} Sep 07 12:58:44.035586 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:44.035385 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7add6a027c5856011bf953250749f26515a9b9f38e620f7d084be5ba15133b98" Sep 07 12:58:48.233743 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:48.233709 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:48.326643 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:48.326611 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:50.041069 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.041036 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9"] Sep 07 12:58:50.041667 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.041437 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ed04487-c991-42ff-a884-1916cae362ba" containerName="pull" Sep 07 12:58:50.041667 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.041454 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ed04487-c991-42ff-a884-1916cae362ba" containerName="pull" Sep 07 12:58:50.041667 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.041467 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ed04487-c991-42ff-a884-1916cae362ba" containerName="extract" Sep 07 12:58:50.041667 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.041475 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ed04487-c991-42ff-a884-1916cae362ba" containerName="extract" Sep 07 12:58:50.041667 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.041489 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ed04487-c991-42ff-a884-1916cae362ba" containerName="util" Sep 07 12:58:50.041667 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.041496 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ed04487-c991-42ff-a884-1916cae362ba" containerName="util" Sep 07 12:58:50.041667 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.041585 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ed04487-c991-42ff-a884-1916cae362ba" containerName="extract" Sep 07 12:58:50.103656 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.103614 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9"] Sep 07 12:58:50.103844 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.103734 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9" Sep 07 12:58:50.105523 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.105492 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kqch\" (UniqueName: \"kubernetes.io/projected/7fdb415c-4cfc-465a-b17f-a17ab8bea0de-kube-api-access-5kqch\") pod \"openshift-lws-operator-fd8ccff4c-zjdp9\" (UID: \"7fdb415c-4cfc-465a-b17f-a17ab8bea0de\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9" Sep 07 12:58:50.105523 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.105515 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 07 12:58:50.105702 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.105533 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fdb415c-4cfc-465a-b17f-a17ab8bea0de-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zjdp9\" (UID: \"7fdb415c-4cfc-465a-b17f-a17ab8bea0de\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9" Sep 07 12:58:50.105702 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.105515 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-fhpjp\"" Sep 07 12:58:50.105836 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.105821 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 07 12:58:50.206766 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.206737 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kqch\" (UniqueName: \"kubernetes.io/projected/7fdb415c-4cfc-465a-b17f-a17ab8bea0de-kube-api-access-5kqch\") pod \"openshift-lws-operator-fd8ccff4c-zjdp9\" (UID: \"7fdb415c-4cfc-465a-b17f-a17ab8bea0de\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9" Sep 07 12:58:50.206766 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.206769 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fdb415c-4cfc-465a-b17f-a17ab8bea0de-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zjdp9\" (UID: \"7fdb415c-4cfc-465a-b17f-a17ab8bea0de\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9" Sep 07 12:58:50.207167 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.207148 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fdb415c-4cfc-465a-b17f-a17ab8bea0de-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zjdp9\" (UID: \"7fdb415c-4cfc-465a-b17f-a17ab8bea0de\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9" Sep 07 12:58:50.215909 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.215886 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kqch\" (UniqueName: \"kubernetes.io/projected/7fdb415c-4cfc-465a-b17f-a17ab8bea0de-kube-api-access-5kqch\") pod \"openshift-lws-operator-fd8ccff4c-zjdp9\" (UID: \"7fdb415c-4cfc-465a-b17f-a17ab8bea0de\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9" Sep 07 12:58:50.413269 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.413198 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9" Sep 07 12:58:50.709003 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:50.708902 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9"] Sep 07 12:58:50.711933 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:58:50.711905 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fdb415c_4cfc_465a_b17f_a17ab8bea0de.slice/crio-db4111a3a48f05ee0847bc8ae4a95493c1ab454a8bc6889b3ab1a2fe4f0f500e WatchSource:0}: Error finding container db4111a3a48f05ee0847bc8ae4a95493c1ab454a8bc6889b3ab1a2fe4f0f500e: Status 404 returned error can't find the container with id db4111a3a48f05ee0847bc8ae4a95493c1ab454a8bc6889b3ab1a2fe4f0f500e Sep 07 12:58:51.065821 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:51.065786 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9" event={"ID":"7fdb415c-4cfc-465a-b17f-a17ab8bea0de","Type":"ContainerStarted","Data":"db4111a3a48f05ee0847bc8ae4a95493c1ab454a8bc6889b3ab1a2fe4f0f500e"} Sep 07 12:58:53.075645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:53.075605 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9" event={"ID":"7fdb415c-4cfc-465a-b17f-a17ab8bea0de","Type":"ContainerStarted","Data":"ea3323b7ffae7b15e90cd2893895fe4e7cdf6a56d16a9a514fadfadee2a4b5b7"} Sep 07 12:58:53.095281 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:53.095237 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjdp9" podStartSLOduration=0.8834675 podStartE2EDuration="3.095221687s" totalImagesPullingTime="2.211754187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:58:53.093062077 +0000 UTC m=+367.123811847" watchObservedRunningTime="2026-09-07 12:58:53.095221687 +0000 UTC m=+367.125971457" Sep 07 12:58:53.845657 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:53.845608 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:53.939336 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:53.939293 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:58:56.429160 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:58:56.429099 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:12.665454 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.665422 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:12.717197 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.717168 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd"] Sep 07 12:59:12.720241 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.720226 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:12.722092 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.722067 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 07 12:59:12.722333 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.722318 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 07 12:59:12.722502 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.722484 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 07 12:59:12.722565 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.722488 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8kkcd\"" Sep 07 12:59:12.728287 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.728266 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd"] Sep 07 12:59:12.754345 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.754310 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:12.778434 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.778403 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxh5s\" (UniqueName: \"kubernetes.io/projected/baf8c2c2-a82c-4bf3-8e0f-2cceffe43731-kube-api-access-sxh5s\") pod \"lws-controller-manager-78895d9f88-9m2vd\" (UID: \"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731\") " pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:12.778581 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.778442 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/baf8c2c2-a82c-4bf3-8e0f-2cceffe43731-manager-config\") pod \"lws-controller-manager-78895d9f88-9m2vd\" (UID: \"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731\") " pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:12.778581 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.778471 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/baf8c2c2-a82c-4bf3-8e0f-2cceffe43731-cert\") pod \"lws-controller-manager-78895d9f88-9m2vd\" (UID: \"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731\") " pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:12.778581 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.778547 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/baf8c2c2-a82c-4bf3-8e0f-2cceffe43731-metrics-cert\") pod \"lws-controller-manager-78895d9f88-9m2vd\" (UID: \"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731\") " pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:12.879798 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.879759 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/baf8c2c2-a82c-4bf3-8e0f-2cceffe43731-manager-config\") pod \"lws-controller-manager-78895d9f88-9m2vd\" (UID: \"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731\") " pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:12.879798 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.879802 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/baf8c2c2-a82c-4bf3-8e0f-2cceffe43731-cert\") pod \"lws-controller-manager-78895d9f88-9m2vd\" (UID: \"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731\") " pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:12.880037 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.879856 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/baf8c2c2-a82c-4bf3-8e0f-2cceffe43731-metrics-cert\") pod \"lws-controller-manager-78895d9f88-9m2vd\" (UID: \"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731\") " pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:12.880037 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.879914 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxh5s\" (UniqueName: \"kubernetes.io/projected/baf8c2c2-a82c-4bf3-8e0f-2cceffe43731-kube-api-access-sxh5s\") pod \"lws-controller-manager-78895d9f88-9m2vd\" (UID: \"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731\") " pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:12.880585 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.880564 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/baf8c2c2-a82c-4bf3-8e0f-2cceffe43731-manager-config\") pod \"lws-controller-manager-78895d9f88-9m2vd\" (UID: \"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731\") " pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:12.882488 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.882460 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/baf8c2c2-a82c-4bf3-8e0f-2cceffe43731-metrics-cert\") pod \"lws-controller-manager-78895d9f88-9m2vd\" (UID: \"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731\") " pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:12.882634 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.882611 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/baf8c2c2-a82c-4bf3-8e0f-2cceffe43731-cert\") pod \"lws-controller-manager-78895d9f88-9m2vd\" (UID: \"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731\") " pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:12.890392 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:12.890370 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxh5s\" (UniqueName: \"kubernetes.io/projected/baf8c2c2-a82c-4bf3-8e0f-2cceffe43731-kube-api-access-sxh5s\") pod \"lws-controller-manager-78895d9f88-9m2vd\" (UID: \"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731\") " pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:13.029100 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:13.029072 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:13.315752 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:13.315672 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd"] Sep 07 12:59:13.318821 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:59:13.318795 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaf8c2c2_a82c_4bf3_8e0f_2cceffe43731.slice/crio-ba5cd4098280d05093f8f92e4d379e01ec0aa0e8d3e82d317ba950cbc21903d8 WatchSource:0}: Error finding container ba5cd4098280d05093f8f92e4d379e01ec0aa0e8d3e82d317ba950cbc21903d8: Status 404 returned error can't find the container with id ba5cd4098280d05093f8f92e4d379e01ec0aa0e8d3e82d317ba950cbc21903d8 Sep 07 12:59:14.157819 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:14.157777 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" event={"ID":"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731","Type":"ContainerStarted","Data":"ba5cd4098280d05093f8f92e4d379e01ec0aa0e8d3e82d317ba950cbc21903d8"} Sep 07 12:59:15.162875 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:15.162843 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" event={"ID":"baf8c2c2-a82c-4bf3-8e0f-2cceffe43731","Type":"ContainerStarted","Data":"eb8b61bfe55fcd2ecd1b3627f0dc9fcf5bdb7bd32061077c73cc92805e5aa17b"} Sep 07 12:59:15.163282 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:15.162957 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:15.180326 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:15.180280 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" podStartSLOduration=1.820739975 podStartE2EDuration="3.18026305s" totalImagesPullingTime="1.359523075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:59:15.179530728 +0000 UTC m=+389.210280499" watchObservedRunningTime="2026-09-07 12:59:15.18026305 +0000 UTC m=+389.211012822" Sep 07 12:59:20.155036 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:20.155001 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:20.257151 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:20.257117 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:20.625832 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:20.625801 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:20.725698 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:20.725666 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:20.824672 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:20.824635 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:20.952311 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:20.952208 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:21.055827 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.055775 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:21.153913 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.153877 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9"] Sep 07 12:59:21.157343 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.157326 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.159377 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.159355 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hz4rq\"" Sep 07 12:59:21.159827 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.159811 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 07 12:59:21.160164 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.160145 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 07 12:59:21.160261 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.160209 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 07 12:59:21.162144 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.162125 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 07 12:59:21.190252 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.190223 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9"] Sep 07 12:59:21.244021 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.243917 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7559daeb-d345-4506-884f-ce03c97f3e3a-manifests\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.244021 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.243952 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7559daeb-d345-4506-884f-ce03c97f3e3a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.244204 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.244067 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7559daeb-d345-4506-884f-ce03c97f3e3a-webhook-cert\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.244204 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.244118 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k77f\" (UniqueName: \"kubernetes.io/projected/7559daeb-d345-4506-884f-ce03c97f3e3a-kube-api-access-7k77f\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.244204 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.244177 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7559daeb-d345-4506-884f-ce03c97f3e3a-tmp\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.328966 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.328930 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:21.345270 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.345236 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7559daeb-d345-4506-884f-ce03c97f3e3a-webhook-cert\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.345270 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.345268 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k77f\" (UniqueName: \"kubernetes.io/projected/7559daeb-d345-4506-884f-ce03c97f3e3a-kube-api-access-7k77f\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.345451 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.345399 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7559daeb-d345-4506-884f-ce03c97f3e3a-tmp\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.345451 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.345440 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7559daeb-d345-4506-884f-ce03c97f3e3a-manifests\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.345552 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.345462 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7559daeb-d345-4506-884f-ce03c97f3e3a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.345871 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.345850 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7559daeb-d345-4506-884f-ce03c97f3e3a-manifests\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.347798 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.347776 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7559daeb-d345-4506-884f-ce03c97f3e3a-tmp\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.347881 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.347843 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7559daeb-d345-4506-884f-ce03c97f3e3a-webhook-cert\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.348113 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.348092 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7559daeb-d345-4506-884f-ce03c97f3e3a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.358335 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.358315 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k77f\" (UniqueName: \"kubernetes.io/projected/7559daeb-d345-4506-884f-ce03c97f3e3a-kube-api-access-7k77f\") pod \"opendatahub-operator-controller-manager-cc4b9478f-7p9s9\" (UID: \"7559daeb-d345-4506-884f-ce03c97f3e3a\") " pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.416705 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.416674 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:21.467645 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.467618 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:21.745910 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:21.745876 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9"] Sep 07 12:59:21.750862 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:59:21.750835 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7559daeb_d345_4506_884f_ce03c97f3e3a.slice/crio-e1bbc4fa131a24691da1562a12e2b8e4b6a1dec58b2ad7fa9c336ce661ae9221 WatchSource:0}: Error finding container e1bbc4fa131a24691da1562a12e2b8e4b6a1dec58b2ad7fa9c336ce661ae9221: Status 404 returned error can't find the container with id e1bbc4fa131a24691da1562a12e2b8e4b6a1dec58b2ad7fa9c336ce661ae9221 Sep 07 12:59:22.192465 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:22.192429 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" event={"ID":"7559daeb-d345-4506-884f-ce03c97f3e3a","Type":"ContainerStarted","Data":"e1bbc4fa131a24691da1562a12e2b8e4b6a1dec58b2ad7fa9c336ce661ae9221"} Sep 07 12:59:22.634565 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:22.634536 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:22.733725 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:22.733689 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:22.834408 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:22.834374 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:26.169578 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:26.169539 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78895d9f88-9m2vd" Sep 07 12:59:26.215633 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:26.215600 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" event={"ID":"7559daeb-d345-4506-884f-ce03c97f3e3a","Type":"ContainerStarted","Data":"5887f5d2dd5a54022223a3d598d9b7ba59a5a84ba2b0753428afaedb844b7e65"} Sep 07 12:59:27.222109 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:27.222010 2576 generic.go:356] "Generic (PLEG): container finished" podID="7559daeb-d345-4506-884f-ce03c97f3e3a" containerID="5887f5d2dd5a54022223a3d598d9b7ba59a5a84ba2b0753428afaedb844b7e65" exitCode=0 Sep 07 12:59:27.222519 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:27.222104 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" event={"ID":"7559daeb-d345-4506-884f-ce03c97f3e3a","Type":"ContainerDied","Data":"5887f5d2dd5a54022223a3d598d9b7ba59a5a84ba2b0753428afaedb844b7e65"} Sep 07 12:59:29.232789 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:29.232753 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" event={"ID":"7559daeb-d345-4506-884f-ce03c97f3e3a","Type":"ContainerStarted","Data":"380e3a9e61441f5f82aecfb1d0643c6613a6f9948eb153a36ddda3494f8c9fc3"} Sep 07 12:59:29.233300 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:29.232871 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:29.258184 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:29.258130 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" podStartSLOduration=1.903033372 podStartE2EDuration="8.258110631s" totalImagesPullingTime="5.267841186s" totalInitContainerRuntime="1.087236073s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:59:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:59:29.256452654 +0000 UTC m=+403.287202424" watchObservedRunningTime="2026-09-07 12:59:29.258110631 +0000 UTC m=+403.288860404" Sep 07 12:59:35.706934 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:35.706897 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:35.774587 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:35.774556 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:40.243682 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:40.243646 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-cc4b9478f-7p9s9" Sep 07 12:59:41.880735 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:41.880691 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:41.984437 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:41.984402 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:42.302306 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:42.302258 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:42.409201 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:42.409174 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:43.581856 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:43.581823 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:43.681955 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:43.681921 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:44.648789 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:44.648752 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:46.637212 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:46.637176 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:46.737861 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:46.737812 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:46.798011 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:46.797963 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:48.138203 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.138165 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:48.238930 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.238889 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:48.806285 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.806255 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs"] Sep 07 12:59:48.812796 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.812776 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:48.814788 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.814764 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-jmng2\"" Sep 07 12:59:48.814882 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.814790 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 07 12:59:48.824137 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.824115 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs"] Sep 07 12:59:48.845571 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.845540 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:48.977194 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.977160 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:48.977194 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.977201 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:48.977379 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.977224 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00293e63-40a8-4d8e-9a0a-37948b8dde30-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:48.977379 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.977254 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7q6b\" (UniqueName: \"kubernetes.io/projected/00293e63-40a8-4d8e-9a0a-37948b8dde30-kube-api-access-r7q6b\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:48.977379 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.977277 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00293e63-40a8-4d8e-9a0a-37948b8dde30-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:48.977379 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.977299 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:48.977379 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.977333 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:48.977541 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.977397 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00293e63-40a8-4d8e-9a0a-37948b8dde30-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:48.977541 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:48.977431 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.044327 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.044286 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:49.078082 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078001 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.078082 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078033 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.078082 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078067 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00293e63-40a8-4d8e-9a0a-37948b8dde30-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.078317 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078099 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7q6b\" (UniqueName: \"kubernetes.io/projected/00293e63-40a8-4d8e-9a0a-37948b8dde30-kube-api-access-r7q6b\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.078317 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078136 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00293e63-40a8-4d8e-9a0a-37948b8dde30-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.078317 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078280 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.078467 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078326 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.078467 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078367 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00293e63-40a8-4d8e-9a0a-37948b8dde30-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.078467 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078409 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.078467 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078408 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.078467 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078460 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.078775 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078728 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.078775 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078737 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.079161 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.078934 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00293e63-40a8-4d8e-9a0a-37948b8dde30-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.080707 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.080685 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00293e63-40a8-4d8e-9a0a-37948b8dde30-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.080802 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.080788 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00293e63-40a8-4d8e-9a0a-37948b8dde30-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.089128 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.089103 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00293e63-40a8-4d8e-9a0a-37948b8dde30-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.089239 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.089221 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7q6b\" (UniqueName: \"kubernetes.io/projected/00293e63-40a8-4d8e-9a0a-37948b8dde30-kube-api-access-r7q6b\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs\" (UID: \"00293e63-40a8-4d8e-9a0a-37948b8dde30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.122231 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.122198 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:49.144019 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.143993 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:49.400954 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:49.400870 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs"] Sep 07 12:59:49.405342 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:59:49.405315 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00293e63_40a8_4d8e_9a0a_37948b8dde30.slice/crio-4a6915d97b0bdecaf618c033ea31abd5099310fe13924a4fc7a9d3371f08c1ba WatchSource:0}: Error finding container 4a6915d97b0bdecaf618c033ea31abd5099310fe13924a4fc7a9d3371f08c1ba: Status 404 returned error can't find the container with id 4a6915d97b0bdecaf618c033ea31abd5099310fe13924a4fc7a9d3371f08c1ba Sep 07 12:59:50.329540 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:50.329509 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" event={"ID":"00293e63-40a8-4d8e-9a0a-37948b8dde30","Type":"ContainerStarted","Data":"4a6915d97b0bdecaf618c033ea31abd5099310fe13924a4fc7a9d3371f08c1ba"} Sep 07 12:59:52.357097 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:52.357061 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 12:59:52.357512 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:52.357148 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 12:59:52.357512 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:52.357193 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 12:59:53.075739 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.075705 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:53.174208 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.174177 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:53.275910 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.275878 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:53.344121 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.344044 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" event={"ID":"00293e63-40a8-4d8e-9a0a-37948b8dde30","Type":"ContainerStarted","Data":"c34c6beec7241825e7e754c7054f1c57657404620d2c3c44a1fbf82838dd1744"} Sep 07 12:59:53.367993 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.364500 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" podStartSLOduration=2.415083041 podStartE2EDuration="5.364480758s" totalImagesPullingTime="2.949397717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:59:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:59:53.363628174 +0000 UTC m=+427.394377938" watchObservedRunningTime="2026-09-07 12:59:53.364480758 +0000 UTC m=+427.395230529" Sep 07 12:59:53.539498 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.539465 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gslld"] Sep 07 12:59:53.542848 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.542833 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gslld" Sep 07 12:59:53.544720 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.544691 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 07 12:59:53.550182 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.550162 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:53.553895 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.553873 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gslld"] Sep 07 12:59:53.649391 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.649301 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:53.721066 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.721034 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 12:59:53.723105 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.723085 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c27675-d34d-4a23-a2c9-87a1b2b50cec-tls-certs\") pod \"model-serving-api-54d89bd5fc-gslld\" (UID: \"87c27675-d34d-4a23-a2c9-87a1b2b50cec\") " pod="opendatahub/model-serving-api-54d89bd5fc-gslld" Sep 07 12:59:53.723177 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.723122 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75fw6\" (UniqueName: \"kubernetes.io/projected/87c27675-d34d-4a23-a2c9-87a1b2b50cec-kube-api-access-75fw6\") pod \"model-serving-api-54d89bd5fc-gslld\" (UID: \"87c27675-d34d-4a23-a2c9-87a1b2b50cec\") " pod="opendatahub/model-serving-api-54d89bd5fc-gslld" Sep 07 12:59:53.823615 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.823583 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c27675-d34d-4a23-a2c9-87a1b2b50cec-tls-certs\") pod \"model-serving-api-54d89bd5fc-gslld\" (UID: \"87c27675-d34d-4a23-a2c9-87a1b2b50cec\") " pod="opendatahub/model-serving-api-54d89bd5fc-gslld" Sep 07 12:59:53.823753 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.823623 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75fw6\" (UniqueName: \"kubernetes.io/projected/87c27675-d34d-4a23-a2c9-87a1b2b50cec-kube-api-access-75fw6\") pod \"model-serving-api-54d89bd5fc-gslld\" (UID: \"87c27675-d34d-4a23-a2c9-87a1b2b50cec\") " pod="opendatahub/model-serving-api-54d89bd5fc-gslld" Sep 07 12:59:53.826047 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.826021 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87c27675-d34d-4a23-a2c9-87a1b2b50cec-tls-certs\") pod \"model-serving-api-54d89bd5fc-gslld\" (UID: \"87c27675-d34d-4a23-a2c9-87a1b2b50cec\") " pod="opendatahub/model-serving-api-54d89bd5fc-gslld" Sep 07 12:59:53.832429 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.832408 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75fw6\" (UniqueName: \"kubernetes.io/projected/87c27675-d34d-4a23-a2c9-87a1b2b50cec-kube-api-access-75fw6\") pod \"model-serving-api-54d89bd5fc-gslld\" (UID: \"87c27675-d34d-4a23-a2c9-87a1b2b50cec\") " pod="opendatahub/model-serving-api-54d89bd5fc-gslld" Sep 07 12:59:53.853336 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:53.853313 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gslld" Sep 07 12:59:54.122830 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:54.122801 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:54.123638 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:54.123609 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gslld"] Sep 07 12:59:54.126741 ip-10-0-142-1 kubenswrapper[2576]: W0907 12:59:54.126715 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c27675_d34d_4a23_a2c9_87a1b2b50cec.slice/crio-a26b20a2fc0e4d0fa534269afa4369832bdb99da75517d851dfb5767b91e402b WatchSource:0}: Error finding container a26b20a2fc0e4d0fa534269afa4369832bdb99da75517d851dfb5767b91e402b: Status 404 returned error can't find the container with id a26b20a2fc0e4d0fa534269afa4369832bdb99da75517d851dfb5767b91e402b Sep 07 12:59:54.128280 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:54.128262 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:54.349234 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:54.349196 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gslld" event={"ID":"87c27675-d34d-4a23-a2c9-87a1b2b50cec","Type":"ContainerStarted","Data":"a26b20a2fc0e4d0fa534269afa4369832bdb99da75517d851dfb5767b91e402b"} Sep 07 12:59:54.349443 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:54.349426 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:54.350333 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:54.350313 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs" Sep 07 12:59:57.365067 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:57.365032 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gslld" event={"ID":"87c27675-d34d-4a23-a2c9-87a1b2b50cec","Type":"ContainerStarted","Data":"1d34e5822bee341ddb6ed1de31c1e25abd2780e49adba92bb11fd52bf52bbbaf"} Sep 07 12:59:57.365472 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:57.365108 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gslld" Sep 07 12:59:57.383142 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:57.383096 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gslld" podStartSLOduration=2.002759752 podStartE2EDuration="4.383082488s" totalImagesPullingTime="2.380322736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:59:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:59:57.381072562 +0000 UTC m=+431.411822342" watchObservedRunningTime="2026-09-07 12:59:57.383082488 +0000 UTC m=+431.413832258" Sep 07 12:59:59.707653 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.707619 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s"] Sep 07 12:59:59.714468 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.714448 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.716776 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.716744 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 07 12:59:59.717366 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.717348 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 07 12:59:59.717725 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.717705 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 07 12:59:59.717954 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.717937 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6pc4q\"" Sep 07 12:59:59.726268 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.726239 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s"] Sep 07 12:59:59.770733 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.770699 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2778977b-4b09-41e3-af74-bc9a05a0a527-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.770891 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.770753 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2778977b-4b09-41e3-af74-bc9a05a0a527-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.770891 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.770802 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2778977b-4b09-41e3-af74-bc9a05a0a527-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.771024 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.770902 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbrbj\" (UniqueName: \"kubernetes.io/projected/2778977b-4b09-41e3-af74-bc9a05a0a527-kube-api-access-vbrbj\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.771024 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.770923 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2778977b-4b09-41e3-af74-bc9a05a0a527-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.771024 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.770946 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2778977b-4b09-41e3-af74-bc9a05a0a527-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.771024 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.770962 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2778977b-4b09-41e3-af74-bc9a05a0a527-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.871352 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.871318 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbrbj\" (UniqueName: \"kubernetes.io/projected/2778977b-4b09-41e3-af74-bc9a05a0a527-kube-api-access-vbrbj\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.871352 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.871354 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2778977b-4b09-41e3-af74-bc9a05a0a527-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.871571 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.871374 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2778977b-4b09-41e3-af74-bc9a05a0a527-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.871571 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.871395 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2778977b-4b09-41e3-af74-bc9a05a0a527-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.871571 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.871432 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2778977b-4b09-41e3-af74-bc9a05a0a527-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.871571 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.871470 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2778977b-4b09-41e3-af74-bc9a05a0a527-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.871571 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.871500 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2778977b-4b09-41e3-af74-bc9a05a0a527-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.872201 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.872177 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2778977b-4b09-41e3-af74-bc9a05a0a527-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.873889 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.873862 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2778977b-4b09-41e3-af74-bc9a05a0a527-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.874453 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.874421 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2778977b-4b09-41e3-af74-bc9a05a0a527-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.874537 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.874451 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2778977b-4b09-41e3-af74-bc9a05a0a527-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.874537 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.874471 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2778977b-4b09-41e3-af74-bc9a05a0a527-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.879946 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.879918 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2778977b-4b09-41e3-af74-bc9a05a0a527-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 12:59:59.880221 ip-10-0-142-1 kubenswrapper[2576]: I0907 12:59:59.880201 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbrbj\" (UniqueName: \"kubernetes.io/projected/2778977b-4b09-41e3-af74-bc9a05a0a527-kube-api-access-vbrbj\") pod \"istiod-openshift-gateway-5677ccb49b-fsw5s\" (UID: \"2778977b-4b09-41e3-af74-bc9a05a0a527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 13:00:00.023686 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:00.023587 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 13:00:00.318997 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:00.318947 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s"] Sep 07 13:00:00.322552 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:00:00.322526 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2778977b_4b09_41e3_af74_bc9a05a0a527.slice/crio-2e1e27fc7f6c45988b0367c7a31669a3a76482aa83b5e1909b55a65da4fd0832 WatchSource:0}: Error finding container 2e1e27fc7f6c45988b0367c7a31669a3a76482aa83b5e1909b55a65da4fd0832: Status 404 returned error can't find the container with id 2e1e27fc7f6c45988b0367c7a31669a3a76482aa83b5e1909b55a65da4fd0832 Sep 07 13:00:00.380548 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:00.380514 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" event={"ID":"2778977b-4b09-41e3-af74-bc9a05a0a527","Type":"ContainerStarted","Data":"2e1e27fc7f6c45988b0367c7a31669a3a76482aa83b5e1909b55a65da4fd0832"} Sep 07 13:00:02.327396 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:02.327356 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:00:02.527942 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:02.527907 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:00:02.528028 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:02.527995 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:00:02.760840 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:02.760805 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:00:02.873579 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:02.873485 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:00:03.396508 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:03.396478 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" event={"ID":"2778977b-4b09-41e3-af74-bc9a05a0a527","Type":"ContainerStarted","Data":"45e5ddc3ee850d13338824be5784b5205921983ea545812877d48a6404b83da4"} Sep 07 13:00:03.396930 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:03.396663 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 13:00:03.398485 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:03.398462 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" Sep 07 13:00:03.419116 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:03.419062 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fsw5s" podStartSLOduration=2.216202513 podStartE2EDuration="4.419045255s" totalImagesPullingTime="2.202842742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:00:03.417044311 +0000 UTC m=+437.447794093" watchObservedRunningTime="2026-09-07 13:00:03.419045255 +0000 UTC m=+437.449795029" Sep 07 13:00:03.533030 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:03.532992 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:00:03.733319 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:03.733235 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:00:08.375415 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:08.375333 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gslld" Sep 07 13:00:48.372241 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:48.372181 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:00:48.432595 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:48.432561 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:00:53.980617 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:53.980575 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:00:54.080833 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:00:54.080795 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:13.552986 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:13.552935 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:16.359543 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:16.359504 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:18.130311 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:18.130269 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:18.435005 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:18.434888 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:18.574110 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:18.574062 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:19.131745 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:19.131710 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:19.845673 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:19.845634 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:20.785804 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:20.785767 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:20.883175 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:20.883135 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:22.153342 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:22.153309 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:22.401229 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:22.401193 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:22.545281 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:22.545244 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:22.647909 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:22.647875 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:25.965783 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:25.965739 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:26.071326 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:26.071292 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:26.743291 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:26.743255 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:29.161848 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:29.161816 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:29.262570 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:29.262539 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:29.596410 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:29.596380 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:30.100829 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:30.100784 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:33.557727 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:33.557691 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:33.659735 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:33.659699 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:33.757436 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:33.757400 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:42.400494 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:42.400408 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:42.518947 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:42.518912 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:45.044194 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:45.044159 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:45.225298 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:45.225263 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:48.226927 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:48.226898 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:48.328263 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:48.328227 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:52.639643 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:52.639609 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:52.739578 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:52.739533 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:52.844240 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:52.844209 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:52.951870 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:52.951779 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:53.849293 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:53.849251 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:01:53.939718 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:01:53.939687 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:01.499505 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:01.499450 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:01.599963 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:01.599926 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:08.528753 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:08.528721 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:08.763847 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:08.763793 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:16.634282 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:16.634245 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:16.727630 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:16.727599 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:22.930374 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:22.930338 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:23.055991 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:23.055931 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:23.136861 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:23.136819 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:23.851238 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:23.851205 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:23.946516 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:23.946482 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:39.434294 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:39.434241 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:39.524200 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:39.524164 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:39.963457 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:39.963424 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:46.413576 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:46.413549 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 13:02:46.416190 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:46.416172 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 13:02:52.808176 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:52.808135 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:02:53.170491 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:02:53.170380 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:01.014767 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:01.014735 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:02.442134 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:02.442099 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:04.931500 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:04.931464 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:05.041292 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:05.041259 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:36.648730 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:36.648689 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:36.757502 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:36.757466 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:38.146492 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:38.146457 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:38.277256 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:38.277211 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:38.669370 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:38.669330 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:39.069163 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.069128 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:39.381257 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.381176 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:39.471408 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.471373 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:39.592006 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.591961 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-9xpdc"] Sep 07 13:03:39.595121 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.595106 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-9xpdc" Sep 07 13:03:39.597173 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.597159 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-46nsv\"" Sep 07 13:03:39.597852 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.597835 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 07 13:03:39.610128 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.610104 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-9xpdc"] Sep 07 13:03:39.682054 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.681941 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j5wf\" (UniqueName: \"kubernetes.io/projected/46f3ff03-1092-43c3-97cf-7c3fdec4dd71-kube-api-access-6j5wf\") pod \"payload-processing-5b54b7fbb7-9xpdc\" (UID: \"46f3ff03-1092-43c3-97cf-7c3fdec4dd71\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9xpdc" Sep 07 13:03:39.682233 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.682106 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46f3ff03-1092-43c3-97cf-7c3fdec4dd71-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-9xpdc\" (UID: \"46f3ff03-1092-43c3-97cf-7c3fdec4dd71\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9xpdc" Sep 07 13:03:39.783496 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.783453 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46f3ff03-1092-43c3-97cf-7c3fdec4dd71-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-9xpdc\" (UID: \"46f3ff03-1092-43c3-97cf-7c3fdec4dd71\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9xpdc" Sep 07 13:03:39.783671 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.783548 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j5wf\" (UniqueName: \"kubernetes.io/projected/46f3ff03-1092-43c3-97cf-7c3fdec4dd71-kube-api-access-6j5wf\") pod \"payload-processing-5b54b7fbb7-9xpdc\" (UID: \"46f3ff03-1092-43c3-97cf-7c3fdec4dd71\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9xpdc" Sep 07 13:03:39.784276 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.784257 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46f3ff03-1092-43c3-97cf-7c3fdec4dd71-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-9xpdc\" (UID: \"46f3ff03-1092-43c3-97cf-7c3fdec4dd71\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9xpdc" Sep 07 13:03:39.795666 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.795641 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j5wf\" (UniqueName: \"kubernetes.io/projected/46f3ff03-1092-43c3-97cf-7c3fdec4dd71-kube-api-access-6j5wf\") pod \"payload-processing-5b54b7fbb7-9xpdc\" (UID: \"46f3ff03-1092-43c3-97cf-7c3fdec4dd71\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9xpdc" Sep 07 13:03:39.905099 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:39.905045 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-9xpdc" Sep 07 13:03:40.184088 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:40.184059 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-9xpdc"] Sep 07 13:03:40.187257 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:03:40.187231 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46f3ff03_1092_43c3_97cf_7c3fdec4dd71.slice/crio-a8defd5c9c6231b4c554186e93f4067034fa22159328a543e84f4dbc177f414d WatchSource:0}: Error finding container a8defd5c9c6231b4c554186e93f4067034fa22159328a543e84f4dbc177f414d: Status 404 returned error can't find the container with id a8defd5c9c6231b4c554186e93f4067034fa22159328a543e84f4dbc177f414d Sep 07 13:03:40.189599 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:40.189580 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 13:03:40.371441 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:40.371410 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-9xpdc" event={"ID":"46f3ff03-1092-43c3-97cf-7c3fdec4dd71","Type":"ContainerStarted","Data":"a8defd5c9c6231b4c554186e93f4067034fa22159328a543e84f4dbc177f414d"} Sep 07 13:03:42.382763 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:42.382728 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-9xpdc" event={"ID":"46f3ff03-1092-43c3-97cf-7c3fdec4dd71","Type":"ContainerStarted","Data":"f9e47453973e5b7f873962fbf95591ed065fd44b8e3c33d3c97676048a64971b"} Sep 07 13:03:42.383176 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:42.382944 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-9xpdc" Sep 07 13:03:42.402957 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:42.402905 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-9xpdc" podStartSLOduration=1.919476137 podStartE2EDuration="3.40289051s" totalImagesPullingTime="1.483414373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:03:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:03:42.401165758 +0000 UTC m=+656.431915521" watchObservedRunningTime="2026-09-07 13:03:42.40289051 +0000 UTC m=+656.433640281" Sep 07 13:03:48.899108 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:48.899072 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:49.000236 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:49.000198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:03:53.389158 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:53.389129 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-9xpdc" Sep 07 13:03:55.551755 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:03:55.551722 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:04:01.471952 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:01.471909 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:04:01.691234 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:01.691191 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:04:06.703478 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.703446 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w"] Sep 07 13:04:06.707095 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.707073 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.709416 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.709394 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 07 13:04:06.709696 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.709680 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 07 13:04:06.709785 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.709722 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 07 13:04:06.709924 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.709904 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4fn2j\"" Sep 07 13:04:06.725194 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.725168 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w"] Sep 07 13:04:06.735009 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.734984 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:04:06.817633 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.817604 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb5743ff-e9ec-48d9-9b89-40357678016f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.817818 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.817663 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb5743ff-e9ec-48d9-9b89-40357678016f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.817818 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.817689 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb5743ff-e9ec-48d9-9b89-40357678016f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.817818 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.817710 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb5743ff-e9ec-48d9-9b89-40357678016f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.817818 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.817728 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqt45\" (UniqueName: \"kubernetes.io/projected/cb5743ff-e9ec-48d9-9b89-40357678016f-kube-api-access-zqt45\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.817964 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.817813 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb5743ff-e9ec-48d9-9b89-40357678016f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.918290 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.918251 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb5743ff-e9ec-48d9-9b89-40357678016f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.918290 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.918292 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb5743ff-e9ec-48d9-9b89-40357678016f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.918523 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.918322 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb5743ff-e9ec-48d9-9b89-40357678016f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.918523 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.918348 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqt45\" (UniqueName: \"kubernetes.io/projected/cb5743ff-e9ec-48d9-9b89-40357678016f-kube-api-access-zqt45\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.918523 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.918459 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb5743ff-e9ec-48d9-9b89-40357678016f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.918523 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.918501 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb5743ff-e9ec-48d9-9b89-40357678016f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.918717 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.918680 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb5743ff-e9ec-48d9-9b89-40357678016f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.918755 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.918733 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb5743ff-e9ec-48d9-9b89-40357678016f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.918836 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.918814 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb5743ff-e9ec-48d9-9b89-40357678016f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.920788 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.920765 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb5743ff-e9ec-48d9-9b89-40357678016f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.921206 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.921186 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb5743ff-e9ec-48d9-9b89-40357678016f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:06.927543 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:06.927519 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqt45\" (UniqueName: \"kubernetes.io/projected/cb5743ff-e9ec-48d9-9b89-40357678016f-kube-api-access-zqt45\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwd2w\" (UID: \"cb5743ff-e9ec-48d9-9b89-40357678016f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:07.017222 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:07.017121 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:07.289255 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:07.289222 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w"] Sep 07 13:04:07.292494 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:04:07.292461 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb5743ff_e9ec_48d9_9b89_40357678016f.slice/crio-1c6b84e117eff33055f6a8d06450135a0d49b19cb99ed182a7bb76d89655a12f WatchSource:0}: Error finding container 1c6b84e117eff33055f6a8d06450135a0d49b19cb99ed182a7bb76d89655a12f: Status 404 returned error can't find the container with id 1c6b84e117eff33055f6a8d06450135a0d49b19cb99ed182a7bb76d89655a12f Sep 07 13:04:07.504928 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:07.504861 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" event={"ID":"cb5743ff-e9ec-48d9-9b89-40357678016f","Type":"ContainerStarted","Data":"1c6b84e117eff33055f6a8d06450135a0d49b19cb99ed182a7bb76d89655a12f"} Sep 07 13:04:08.510872 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:08.510830 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" event={"ID":"cb5743ff-e9ec-48d9-9b89-40357678016f","Type":"ContainerStarted","Data":"a65a03d361913af644c73b10d1c9419834d8b602ead5c283f927edc23b0059ce"} Sep 07 13:04:08.531341 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:08.531264 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" podStartSLOduration=1.536351933 podStartE2EDuration="2.53124959s" totalImagesPullingTime="994.897657ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:04:08.529652763 +0000 UTC m=+682.560402545" watchObservedRunningTime="2026-09-07 13:04:08.53124959 +0000 UTC m=+682.561999360" Sep 07 13:04:16.419865 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:16.419819 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:04:17.017892 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:17.017848 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:17.018100 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:17.017922 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:17.030631 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:17.030605 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:17.560894 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:17.560861 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwd2w" Sep 07 13:04:25.907276 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:25.907234 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l"] Sep 07 13:04:25.912074 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:25.912051 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:25.914031 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:25.914010 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 07 13:04:25.920920 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:25.920900 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l"] Sep 07 13:04:25.939738 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:25.939708 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:04:25.987820 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:25.987790 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a4b5ed5-8675-4198-a361-86c316cd4578-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:25.987948 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:25.987829 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a4b5ed5-8675-4198-a361-86c316cd4578-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:25.987948 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:25.987865 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a4b5ed5-8675-4198-a361-86c316cd4578-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:25.988064 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:25.987953 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a4b5ed5-8675-4198-a361-86c316cd4578-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:25.988064 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:25.988005 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn922\" (UniqueName: \"kubernetes.io/projected/2a4b5ed5-8675-4198-a361-86c316cd4578-kube-api-access-zn922\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:25.988138 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:25.988063 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a4b5ed5-8675-4198-a361-86c316cd4578-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.088992 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.088927 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a4b5ed5-8675-4198-a361-86c316cd4578-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.088992 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.088996 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a4b5ed5-8675-4198-a361-86c316cd4578-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.089223 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.089022 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a4b5ed5-8675-4198-a361-86c316cd4578-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.089223 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.089059 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a4b5ed5-8675-4198-a361-86c316cd4578-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.089223 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.089078 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn922\" (UniqueName: \"kubernetes.io/projected/2a4b5ed5-8675-4198-a361-86c316cd4578-kube-api-access-zn922\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.089223 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.089112 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a4b5ed5-8675-4198-a361-86c316cd4578-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.089514 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.089492 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a4b5ed5-8675-4198-a361-86c316cd4578-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.089595 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.089538 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a4b5ed5-8675-4198-a361-86c316cd4578-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.089595 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.089566 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a4b5ed5-8675-4198-a361-86c316cd4578-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.091515 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.091492 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a4b5ed5-8675-4198-a361-86c316cd4578-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.091665 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.091650 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a4b5ed5-8675-4198-a361-86c316cd4578-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.097631 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.097609 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn922\" (UniqueName: \"kubernetes.io/projected/2a4b5ed5-8675-4198-a361-86c316cd4578-kube-api-access-zn922\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2pk6l\" (UID: \"2a4b5ed5-8675-4198-a361-86c316cd4578\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.223725 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.223642 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:26.498604 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.498503 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l"] Sep 07 13:04:26.500848 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:04:26.500815 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a4b5ed5_8675_4198_a361_86c316cd4578.slice/crio-1e5a04b4b2f781711c5d15a71fc4585cdfa50c45c55d35e237c7a731e60ed1b5 WatchSource:0}: Error finding container 1e5a04b4b2f781711c5d15a71fc4585cdfa50c45c55d35e237c7a731e60ed1b5: Status 404 returned error can't find the container with id 1e5a04b4b2f781711c5d15a71fc4585cdfa50c45c55d35e237c7a731e60ed1b5 Sep 07 13:04:26.598086 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:26.598050 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" event={"ID":"2a4b5ed5-8675-4198-a361-86c316cd4578","Type":"ContainerStarted","Data":"1e5a04b4b2f781711c5d15a71fc4585cdfa50c45c55d35e237c7a731e60ed1b5"} Sep 07 13:04:27.603958 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:27.603926 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" event={"ID":"2a4b5ed5-8675-4198-a361-86c316cd4578","Type":"ContainerStarted","Data":"f3e2563941631d48b584de086c2d1fb6131a76f0061e1438c2626ec2c0a182a5"} Sep 07 13:04:27.628842 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:27.628760 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" podStartSLOduration=2.441971485 podStartE2EDuration="2.628741326s" totalImagesPullingTime="186.769841ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:04:27.628118339 +0000 UTC m=+701.658868109" watchObservedRunningTime="2026-09-07 13:04:27.628741326 +0000 UTC m=+701.659491097" Sep 07 13:04:32.421188 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.421157 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc"] Sep 07 13:04:32.424486 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.424465 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.426499 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.426476 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 07 13:04:32.437116 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.437094 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc"] Sep 07 13:04:32.465651 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.465621 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:04:32.546467 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.546431 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.546467 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.546464 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcgrw\" (UniqueName: \"kubernetes.io/projected/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-kube-api-access-mcgrw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.546675 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.546496 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.546675 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.546569 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.546675 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.546625 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.546786 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.546691 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.648034 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.648002 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.648216 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.648050 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.648216 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.648080 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.648216 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.648118 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.648216 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.648135 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcgrw\" (UniqueName: \"kubernetes.io/projected/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-kube-api-access-mcgrw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.648216 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.648173 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.648505 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.648480 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.648569 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.648526 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.648678 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.648660 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.650539 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.650521 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.650817 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.650796 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.661308 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.661279 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcgrw\" (UniqueName: \"kubernetes.io/projected/1d7ef80b-b643-4043-8b8c-7bbd71abc95b-kube-api-access-mcgrw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc\" (UID: \"1d7ef80b-b643-4043-8b8c-7bbd71abc95b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:32.735489 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:32.735398 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:33.014378 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:33.014298 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc"] Sep 07 13:04:33.018331 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:04:33.018307 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d7ef80b_b643_4043_8b8c_7bbd71abc95b.slice/crio-417de94e9d6beea0038bdb1c0a2f4657d09fde4cf2a6732d969a0b4bb3927d82 WatchSource:0}: Error finding container 417de94e9d6beea0038bdb1c0a2f4657d09fde4cf2a6732d969a0b4bb3927d82: Status 404 returned error can't find the container with id 417de94e9d6beea0038bdb1c0a2f4657d09fde4cf2a6732d969a0b4bb3927d82 Sep 07 13:04:33.632952 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:33.632915 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" event={"ID":"1d7ef80b-b643-4043-8b8c-7bbd71abc95b","Type":"ContainerStarted","Data":"843904fdb294f9466330a0001ea58f1bd4c8bcb9dd7e4271186bfcf8f98e1836"} Sep 07 13:04:33.633392 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:33.632966 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" event={"ID":"1d7ef80b-b643-4043-8b8c-7bbd71abc95b","Type":"ContainerStarted","Data":"417de94e9d6beea0038bdb1c0a2f4657d09fde4cf2a6732d969a0b4bb3927d82"} Sep 07 13:04:33.655783 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:33.655737 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" podStartSLOduration=1.483560397 podStartE2EDuration="1.655719811s" totalImagesPullingTime="172.159414ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:04:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:04:33.654134963 +0000 UTC m=+707.684884744" watchObservedRunningTime="2026-09-07 13:04:33.655719811 +0000 UTC m=+707.686469582" Sep 07 13:04:36.224723 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:36.224676 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:36.225144 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:36.224773 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:36.237221 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:36.237196 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:36.656363 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:36.656338 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2pk6l" Sep 07 13:04:41.338438 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:41.338340 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:04:42.735777 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:42.735727 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:42.736245 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:42.735797 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:42.748543 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:42.748519 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:43.689253 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:43.689227 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc" Sep 07 13:04:47.934593 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:47.934542 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:04:50.526498 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:50.526456 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:04:55.636456 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:55.636423 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:04:55.738714 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:04:55.738677 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:05:31.346065 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:05:31.346033 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:05:56.074259 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:05:56.074220 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:05:56.174351 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:05:56.174294 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:09.095691 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:09.095581 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:09.226427 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:09.226389 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:10.204830 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:10.204793 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:10.297394 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:10.297368 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:25.453989 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:25.453928 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:25.596988 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:25.596934 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:25.815614 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:25.815577 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:26.160753 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:26.160645 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:26.888493 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:26.888462 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:26.987021 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:26.986960 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:27.379862 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:27.379826 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:29.364662 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:29.364630 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:29.463255 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:29.463215 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:30.652462 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:30.652424 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:30.729672 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:30.729632 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:30.796833 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:30.796793 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:31.222363 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.222322 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:31.303725 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.303691 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:31.337759 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.337735 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv"] Sep 07 13:06:31.341126 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.341111 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" Sep 07 13:06:31.343246 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.343224 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-7ce94d-dockercfg-wn552\"" Sep 07 13:06:31.343812 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.343794 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-7ce94d\"" Sep 07 13:06:31.356463 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.356442 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv"] Sep 07 13:06:31.358108 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.358087 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm"] Sep 07 13:06:31.361247 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.361230 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" Sep 07 13:06:31.372731 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.372709 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm"] Sep 07 13:06:31.467934 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.467902 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7866a05-0fee-4b7e-93de-3e6512df788c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm\" (UID: \"f7866a05-0fee-4b7e-93de-3e6512df788c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" Sep 07 13:06:31.468146 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.467987 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4gjw\" (UniqueName: \"kubernetes.io/projected/64f34894-7c4f-4bcb-832d-d92aeef8710f-kube-api-access-q4gjw\") pod \"payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv\" (UID: \"64f34894-7c4f-4bcb-832d-d92aeef8710f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" Sep 07 13:06:31.468146 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.468047 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjv76\" (UniqueName: \"kubernetes.io/projected/f7866a05-0fee-4b7e-93de-3e6512df788c-kube-api-access-bjv76\") pod \"payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm\" (UID: \"f7866a05-0fee-4b7e-93de-3e6512df788c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" Sep 07 13:06:31.468146 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.468093 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64f34894-7c4f-4bcb-832d-d92aeef8710f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv\" (UID: \"64f34894-7c4f-4bcb-832d-d92aeef8710f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" Sep 07 13:06:31.568629 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.568591 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4gjw\" (UniqueName: \"kubernetes.io/projected/64f34894-7c4f-4bcb-832d-d92aeef8710f-kube-api-access-q4gjw\") pod \"payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv\" (UID: \"64f34894-7c4f-4bcb-832d-d92aeef8710f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" Sep 07 13:06:31.568837 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.568637 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjv76\" (UniqueName: \"kubernetes.io/projected/f7866a05-0fee-4b7e-93de-3e6512df788c-kube-api-access-bjv76\") pod \"payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm\" (UID: \"f7866a05-0fee-4b7e-93de-3e6512df788c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" Sep 07 13:06:31.568837 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.568662 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64f34894-7c4f-4bcb-832d-d92aeef8710f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv\" (UID: \"64f34894-7c4f-4bcb-832d-d92aeef8710f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" Sep 07 13:06:31.568945 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.568859 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7866a05-0fee-4b7e-93de-3e6512df788c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm\" (UID: \"f7866a05-0fee-4b7e-93de-3e6512df788c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" Sep 07 13:06:31.569397 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.569378 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64f34894-7c4f-4bcb-832d-d92aeef8710f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv\" (UID: \"64f34894-7c4f-4bcb-832d-d92aeef8710f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" Sep 07 13:06:31.569580 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.569562 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7866a05-0fee-4b7e-93de-3e6512df788c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm\" (UID: \"f7866a05-0fee-4b7e-93de-3e6512df788c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" Sep 07 13:06:31.577722 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.577700 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4gjw\" (UniqueName: \"kubernetes.io/projected/64f34894-7c4f-4bcb-832d-d92aeef8710f-kube-api-access-q4gjw\") pod \"payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv\" (UID: \"64f34894-7c4f-4bcb-832d-d92aeef8710f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" Sep 07 13:06:31.577852 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.577755 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjv76\" (UniqueName: \"kubernetes.io/projected/f7866a05-0fee-4b7e-93de-3e6512df788c-kube-api-access-bjv76\") pod \"payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm\" (UID: \"f7866a05-0fee-4b7e-93de-3e6512df788c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" Sep 07 13:06:31.651234 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.651205 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" Sep 07 13:06:31.671261 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.671238 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" Sep 07 13:06:31.929375 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.929351 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv"] Sep 07 13:06:31.932571 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:06:31.932544 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f34894_7c4f_4bcb_832d_d92aeef8710f.slice/crio-0289f6c52c932bbe08ce89ecd18775b891b5e94a70baf47196a729fb6ec4eb66 WatchSource:0}: Error finding container 0289f6c52c932bbe08ce89ecd18775b891b5e94a70baf47196a729fb6ec4eb66: Status 404 returned error can't find the container with id 0289f6c52c932bbe08ce89ecd18775b891b5e94a70baf47196a729fb6ec4eb66 Sep 07 13:06:31.956045 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:31.956022 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm"] Sep 07 13:06:31.959937 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:06:31.959915 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7866a05_0fee_4b7e_93de_3e6512df788c.slice/crio-78bdf28c065a430fe5f68a8f76d10d2f53fe44c6c51468f695e33681398e146e WatchSource:0}: Error finding container 78bdf28c065a430fe5f68a8f76d10d2f53fe44c6c51468f695e33681398e146e: Status 404 returned error can't find the container with id 78bdf28c065a430fe5f68a8f76d10d2f53fe44c6c51468f695e33681398e146e Sep 07 13:06:32.183935 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:32.183834 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" event={"ID":"f7866a05-0fee-4b7e-93de-3e6512df788c","Type":"ContainerStarted","Data":"6ab4e874f10bec38a70310852d2a68cca6799d42cfdbb1d587fcf3e1b5eb7ed1"} Sep 07 13:06:32.183935 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:32.183898 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" event={"ID":"f7866a05-0fee-4b7e-93de-3e6512df788c","Type":"ContainerStarted","Data":"78bdf28c065a430fe5f68a8f76d10d2f53fe44c6c51468f695e33681398e146e"} Sep 07 13:06:32.183935 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:32.183930 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" Sep 07 13:06:32.185553 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:32.185527 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" event={"ID":"64f34894-7c4f-4bcb-832d-d92aeef8710f","Type":"ContainerStarted","Data":"b32fa950b36a7cdb4802ab3126b6d61fb4603b741efe5773d340fc5965738952"} Sep 07 13:06:32.185658 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:32.185570 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" event={"ID":"64f34894-7c4f-4bcb-832d-d92aeef8710f","Type":"ContainerStarted","Data":"0289f6c52c932bbe08ce89ecd18775b891b5e94a70baf47196a729fb6ec4eb66"} Sep 07 13:06:32.185658 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:32.185612 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" Sep 07 13:06:32.204203 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:32.204161 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" podStartSLOduration=1.204146478 podStartE2EDuration="1.204146478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:06:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:06:32.201907498 +0000 UTC m=+826.232657286" watchObservedRunningTime="2026-09-07 13:06:32.204146478 +0000 UTC m=+826.234896240" Sep 07 13:06:32.222079 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:32.222037 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" podStartSLOduration=1.222023632 podStartE2EDuration="1.222023632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:06:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:06:32.219914554 +0000 UTC m=+826.250664324" watchObservedRunningTime="2026-09-07 13:06:32.222023632 +0000 UTC m=+826.252773402" Sep 07 13:06:33.997129 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:33.997085 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:34.263711 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.263624 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:34.363628 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.363594 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:34.598013 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.594320 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7"] Sep 07 13:06:34.599391 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.599366 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.601714 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.601690 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2cd995c1-openshift-default-dockercfg-2mnnr\"" Sep 07 13:06:34.609344 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.609317 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7"] Sep 07 13:06:34.620235 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.620204 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:34.699842 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.699807 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cxgv\" (UniqueName: \"kubernetes.io/projected/01a7622f-e712-45b4-a6aa-74e827350a4c-kube-api-access-5cxgv\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.700043 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.699871 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-workload-socket\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.700043 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.699920 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-token\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.700043 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.699964 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-podinfo\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.700043 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.700017 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a7622f-e712-45b4-a6aa-74e827350a4c-istiod-ca-cert\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.700197 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.700072 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-envoy\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.700197 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.700101 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-workload-certs\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.700197 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.700135 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-credential-socket\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.700197 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.700163 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-data\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.720564 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.720532 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:34.800907 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.800868 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-data\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.801120 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.800946 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cxgv\" (UniqueName: \"kubernetes.io/projected/01a7622f-e712-45b4-a6aa-74e827350a4c-kube-api-access-5cxgv\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.801120 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.801070 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-workload-socket\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.801120 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.801095 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-token\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.801288 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.801127 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-podinfo\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.801288 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.801164 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a7622f-e712-45b4-a6aa-74e827350a4c-istiod-ca-cert\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.801288 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.801200 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-envoy\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.801288 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.801246 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-workload-certs\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.801288 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.801274 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-credential-socket\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.801553 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.801328 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-data\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.801553 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.801462 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-workload-socket\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.801658 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.801643 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-credential-socket\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.801730 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.801708 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-workload-certs\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.802022 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.802000 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a7622f-e712-45b4-a6aa-74e827350a4c-istiod-ca-cert\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.803614 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.803586 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-envoy\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.803912 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.803892 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-podinfo\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.809838 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.809811 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cxgv\" (UniqueName: \"kubernetes.io/projected/01a7622f-e712-45b4-a6aa-74e827350a4c-kube-api-access-5cxgv\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.810077 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.810059 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-token\") pod \"e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:34.911845 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:34.911760 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:35.194570 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:35.194542 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7"] Sep 07 13:06:35.195786 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:06:35.195759 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a7622f_e712_45b4_a6aa_74e827350a4c.slice/crio-e1d47c8a29d169ae6d30ba14fffab98eb6a7adf3327910cf7bc88142dd10fc03 WatchSource:0}: Error finding container e1d47c8a29d169ae6d30ba14fffab98eb6a7adf3327910cf7bc88142dd10fc03: Status 404 returned error can't find the container with id e1d47c8a29d169ae6d30ba14fffab98eb6a7adf3327910cf7bc88142dd10fc03 Sep 07 13:06:35.198176 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:35.198138 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:06:35.198275 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:35.198222 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:06:35.198275 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:35.198265 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:06:35.206190 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:35.206169 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" event={"ID":"01a7622f-e712-45b4-a6aa-74e827350a4c","Type":"ContainerStarted","Data":"e1d47c8a29d169ae6d30ba14fffab98eb6a7adf3327910cf7bc88142dd10fc03"} Sep 07 13:06:35.368335 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:35.368300 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:35.684232 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:35.684193 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:35.815637 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:35.815606 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:36.213107 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:36.213076 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" event={"ID":"01a7622f-e712-45b4-a6aa-74e827350a4c","Type":"ContainerStarted","Data":"6ca1ab146a5d53de87f634c17069d668779bbc24c0127cfc772a374999d17520"} Sep 07 13:06:36.260393 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:36.260346 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" podStartSLOduration=2.260330834 podStartE2EDuration="2.260330834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:06:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:06:36.254350104 +0000 UTC m=+830.285099875" watchObservedRunningTime="2026-09-07 13:06:36.260330834 +0000 UTC m=+830.291080604" Sep 07 13:06:36.911938 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:36.911902 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:36.916925 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:36.916899 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:37.218530 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:37.218450 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:37.219552 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:37.219532 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:37.686863 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:37.686832 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:37.777362 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:37.777325 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:38.782492 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:38.782450 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:38.884759 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:38.884722 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:39.435413 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:39.435373 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:39.630625 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:39.630592 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:39.807663 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:39.807620 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:39.895350 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:39.895313 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:40.998346 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:40.998311 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:41.098840 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.098808 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:41.199444 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.199407 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:41.527431 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.527374 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:41.627568 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.627528 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:41.650855 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.650827 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds"] Sep 07 13:06:41.654121 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.654103 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" Sep 07 13:06:41.656025 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.656005 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-e9d55a\"" Sep 07 13:06:41.656120 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.656005 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-e9d55a-dockercfg-xh2q4\"" Sep 07 13:06:41.667126 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.667105 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds"] Sep 07 13:06:41.698948 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.698928 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s"] Sep 07 13:06:41.702253 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.702239 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" Sep 07 13:06:41.714530 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.714506 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s"] Sep 07 13:06:41.751602 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.751571 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s\" (UID: \"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" Sep 07 13:06:41.751763 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.751742 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbhww\" (UniqueName: \"kubernetes.io/projected/fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4-kube-api-access-kbhww\") pod \"payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s\" (UID: \"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" Sep 07 13:06:41.853659 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.853527 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s\" (UID: \"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" Sep 07 13:06:41.854560 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.854519 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cebf60da-b7cb-48cc-9336-535074a4ccc0-plugins-config-volume\") pod \"payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds\" (UID: \"cebf60da-b7cb-48cc-9336-535074a4ccc0\") " pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" Sep 07 13:06:41.854677 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.854572 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52mqm\" (UniqueName: \"kubernetes.io/projected/cebf60da-b7cb-48cc-9336-535074a4ccc0-kube-api-access-52mqm\") pod \"payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds\" (UID: \"cebf60da-b7cb-48cc-9336-535074a4ccc0\") " pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" Sep 07 13:06:41.854677 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.854612 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbhww\" (UniqueName: \"kubernetes.io/projected/fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4-kube-api-access-kbhww\") pod \"payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s\" (UID: \"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" Sep 07 13:06:41.855205 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.855177 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s\" (UID: \"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" Sep 07 13:06:41.864008 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.863958 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbhww\" (UniqueName: \"kubernetes.io/projected/fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4-kube-api-access-kbhww\") pod \"payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s\" (UID: \"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" Sep 07 13:06:41.956141 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.956108 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cebf60da-b7cb-48cc-9336-535074a4ccc0-plugins-config-volume\") pod \"payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds\" (UID: \"cebf60da-b7cb-48cc-9336-535074a4ccc0\") " pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" Sep 07 13:06:41.956323 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.956148 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52mqm\" (UniqueName: \"kubernetes.io/projected/cebf60da-b7cb-48cc-9336-535074a4ccc0-kube-api-access-52mqm\") pod \"payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds\" (UID: \"cebf60da-b7cb-48cc-9336-535074a4ccc0\") " pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" Sep 07 13:06:41.956770 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.956749 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cebf60da-b7cb-48cc-9336-535074a4ccc0-plugins-config-volume\") pod \"payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds\" (UID: \"cebf60da-b7cb-48cc-9336-535074a4ccc0\") " pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" Sep 07 13:06:41.968008 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:41.967963 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52mqm\" (UniqueName: \"kubernetes.io/projected/cebf60da-b7cb-48cc-9336-535074a4ccc0-kube-api-access-52mqm\") pod \"payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds\" (UID: \"cebf60da-b7cb-48cc-9336-535074a4ccc0\") " pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" Sep 07 13:06:42.011199 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:42.011167 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" Sep 07 13:06:42.153475 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:42.153382 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:42.264656 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:42.264626 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" Sep 07 13:06:42.293651 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:42.293624 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s"] Sep 07 13:06:42.296142 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:06:42.296116 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd73fb5d_a518_4e41_b7ff_b6a37fceb1a4.slice/crio-ada20e4cdec0d1e4bcd15c02d6bdc77c4320636708b1ac2a77f1ec72ebddb0ea WatchSource:0}: Error finding container ada20e4cdec0d1e4bcd15c02d6bdc77c4320636708b1ac2a77f1ec72ebddb0ea: Status 404 returned error can't find the container with id ada20e4cdec0d1e4bcd15c02d6bdc77c4320636708b1ac2a77f1ec72ebddb0ea Sep 07 13:06:42.544080 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:42.544055 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds"] Sep 07 13:06:42.545618 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:06:42.545594 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcebf60da_b7cb_48cc_9336_535074a4ccc0.slice/crio-4e699b7bd224fe8891e94a4f0c6b77118cf391ca07b5e6195819bbaa4b42f02c WatchSource:0}: Error finding container 4e699b7bd224fe8891e94a4f0c6b77118cf391ca07b5e6195819bbaa4b42f02c: Status 404 returned error can't find the container with id 4e699b7bd224fe8891e94a4f0c6b77118cf391ca07b5e6195819bbaa4b42f02c Sep 07 13:06:42.886227 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:42.886142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:43.197076 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:43.196989 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" Sep 07 13:06:43.197459 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:43.197172 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" Sep 07 13:06:43.248703 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:43.248666 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" event={"ID":"cebf60da-b7cb-48cc-9336-535074a4ccc0","Type":"ContainerStarted","Data":"5eaf366774a1be8baac20044e96a2faa5476e32ec60e9c58d42dbec02f63e6ba"} Sep 07 13:06:43.248886 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:43.248729 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" event={"ID":"cebf60da-b7cb-48cc-9336-535074a4ccc0","Type":"ContainerStarted","Data":"4e699b7bd224fe8891e94a4f0c6b77118cf391ca07b5e6195819bbaa4b42f02c"} Sep 07 13:06:43.248886 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:43.248770 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" Sep 07 13:06:43.250526 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:43.250500 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" event={"ID":"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4","Type":"ContainerStarted","Data":"d82c2f7da9863103294f2d989d15b26fada20a19a45a43fe899261a6ecc12dd7"} Sep 07 13:06:43.250673 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:43.250545 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" event={"ID":"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4","Type":"ContainerStarted","Data":"ada20e4cdec0d1e4bcd15c02d6bdc77c4320636708b1ac2a77f1ec72ebddb0ea"} Sep 07 13:06:43.250673 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:43.250575 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" Sep 07 13:06:43.265340 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:43.265247 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" podStartSLOduration=2.265231462 podStartE2EDuration="2.265231462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:06:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:06:43.264106429 +0000 UTC m=+837.294856199" watchObservedRunningTime="2026-09-07 13:06:43.265231462 +0000 UTC m=+837.295981233" Sep 07 13:06:43.287364 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:43.287315 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" podStartSLOduration=2.287296936 podStartE2EDuration="2.287296936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:06:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:06:43.285340865 +0000 UTC m=+837.316090648" watchObservedRunningTime="2026-09-07 13:06:43.287296936 +0000 UTC m=+837.318046707" Sep 07 13:06:46.170887 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:46.170849 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:46.321583 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:46.321543 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:46.423119 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:46.423034 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:46.524336 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:46.524306 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:47.524080 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:47.524041 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:49.254918 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:49.254886 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:50.927045 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:50.927001 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:51.749788 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:51.749759 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:53.201132 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:53.201099 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:53.300443 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:53.300403 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:53.455866 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:53.455780 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:54.257684 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:54.257654 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" Sep 07 13:06:54.258047 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:54.257726 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" Sep 07 13:06:54.261269 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:54.261228 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:54.588901 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:54.588848 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:54.909040 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:54.908932 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:55.530578 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:55.530525 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:56.325608 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:56.325575 2576 generic.go:356] "Generic (PLEG): container finished" podID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerID="6ca1ab146a5d53de87f634c17069d668779bbc24c0127cfc772a374999d17520" exitCode=0 Sep 07 13:06:56.325820 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:56.325653 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" event={"ID":"01a7622f-e712-45b4-a6aa-74e827350a4c","Type":"ContainerDied","Data":"6ca1ab146a5d53de87f634c17069d668779bbc24c0127cfc772a374999d17520"} Sep 07 13:06:56.326045 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:56.326032 2576 scope.go:117] "RemoveContainer" containerID="6ca1ab146a5d53de87f634c17069d668779bbc24c0127cfc772a374999d17520" Sep 07 13:06:56.327068 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:56.327044 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:06:56.327140 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:56.327105 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:06:56.327187 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:56.327138 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:06:57.332126 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:57.332091 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" event={"ID":"01a7622f-e712-45b4-a6aa-74e827350a4c","Type":"ContainerStarted","Data":"f574aaa9e350ae30961e192b5342197f86c106d3d5d36342a73a0939bcf5c6ae"} Sep 07 13:06:57.911928 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:57.911883 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:06:57.912167 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:57.912144 2576 patch_prober.go:28] interesting pod/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Sep 07 13:06:57.912217 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:57.912191 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Sep 07 13:06:58.601848 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:58.601806 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:58.807560 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:58.807521 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:58.912398 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:58.912304 2576 patch_prober.go:28] interesting pod/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Sep 07 13:06:58.912398 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:58.912366 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Sep 07 13:06:59.779998 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:59.767303 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:59.866410 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:59.866362 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:06:59.912223 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:59.912190 2576 patch_prober.go:28] interesting pod/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Sep 07 13:06:59.912388 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:06:59.912248 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Sep 07 13:07:00.280145 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.280104 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:00.473579 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.473540 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299"] Sep 07 13:07:00.478231 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.478204 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" Sep 07 13:07:00.482448 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.480825 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-76f028-dockercfg-7ff58\"" Sep 07 13:07:00.482448 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.482189 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-76f028\"" Sep 07 13:07:00.494490 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.494464 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299"] Sep 07 13:07:00.524355 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.524315 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64"] Sep 07 13:07:00.527856 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.527832 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" Sep 07 13:07:00.531113 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.531045 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cf3a9c5-c74f-4a9b-8651-a31efff94c8d-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299\" (UID: \"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" Sep 07 13:07:00.531113 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.531106 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgpwq\" (UniqueName: \"kubernetes.io/projected/9cf3a9c5-c74f-4a9b-8651-a31efff94c8d-kube-api-access-pgpwq\") pod \"payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299\" (UID: \"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" Sep 07 13:07:00.538301 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.538279 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64"] Sep 07 13:07:00.632495 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.632460 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cf3a9c5-c74f-4a9b-8651-a31efff94c8d-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299\" (UID: \"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" Sep 07 13:07:00.632660 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.632520 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgpwq\" (UniqueName: \"kubernetes.io/projected/9cf3a9c5-c74f-4a9b-8651-a31efff94c8d-kube-api-access-pgpwq\") pod \"payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299\" (UID: \"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" Sep 07 13:07:00.632660 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.632552 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcvjh\" (UniqueName: \"kubernetes.io/projected/b19d504b-b304-431e-ac80-3c4e316c8063-kube-api-access-dcvjh\") pod \"payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64\" (UID: \"b19d504b-b304-431e-ac80-3c4e316c8063\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" Sep 07 13:07:00.632660 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.632576 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b19d504b-b304-431e-ac80-3c4e316c8063-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64\" (UID: \"b19d504b-b304-431e-ac80-3c4e316c8063\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" Sep 07 13:07:00.633203 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.633184 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cf3a9c5-c74f-4a9b-8651-a31efff94c8d-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299\" (UID: \"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" Sep 07 13:07:00.647683 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.647650 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgpwq\" (UniqueName: \"kubernetes.io/projected/9cf3a9c5-c74f-4a9b-8651-a31efff94c8d-kube-api-access-pgpwq\") pod \"payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299\" (UID: \"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" Sep 07 13:07:00.733391 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.733355 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcvjh\" (UniqueName: \"kubernetes.io/projected/b19d504b-b304-431e-ac80-3c4e316c8063-kube-api-access-dcvjh\") pod \"payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64\" (UID: \"b19d504b-b304-431e-ac80-3c4e316c8063\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" Sep 07 13:07:00.733391 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.733395 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b19d504b-b304-431e-ac80-3c4e316c8063-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64\" (UID: \"b19d504b-b304-431e-ac80-3c4e316c8063\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" Sep 07 13:07:00.734071 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.734051 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b19d504b-b304-431e-ac80-3c4e316c8063-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64\" (UID: \"b19d504b-b304-431e-ac80-3c4e316c8063\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" Sep 07 13:07:00.742749 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.742725 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcvjh\" (UniqueName: \"kubernetes.io/projected/b19d504b-b304-431e-ac80-3c4e316c8063-kube-api-access-dcvjh\") pod \"payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64\" (UID: \"b19d504b-b304-431e-ac80-3c4e316c8063\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" Sep 07 13:07:00.798767 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.798681 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" Sep 07 13:07:00.839658 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.839622 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" Sep 07 13:07:00.912431 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.912403 2576 patch_prober.go:28] interesting pod/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Sep 07 13:07:00.912575 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:00.912461 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Sep 07 13:07:01.079708 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.079588 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299"] Sep 07 13:07:01.100603 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.100526 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:01.145991 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.145872 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64"] Sep 07 13:07:01.148137 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:07:01.148108 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb19d504b_b304_431e_ac80_3c4e316c8063.slice/crio-6e7b775e3575025f0a3dcf422ab4f326db84798f558deb507cb8bc02f15a5210 WatchSource:0}: Error finding container 6e7b775e3575025f0a3dcf422ab4f326db84798f558deb507cb8bc02f15a5210: Status 404 returned error can't find the container with id 6e7b775e3575025f0a3dcf422ab4f326db84798f558deb507cb8bc02f15a5210 Sep 07 13:07:01.177997 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.177789 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:01.355276 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.355146 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" event={"ID":"b19d504b-b304-431e-ac80-3c4e316c8063","Type":"ContainerStarted","Data":"aef69b99baf106a92849d6c1e8724b5a9924dae557cffac0614d128e365b7981"} Sep 07 13:07:01.355276 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.355196 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" event={"ID":"b19d504b-b304-431e-ac80-3c4e316c8063","Type":"ContainerStarted","Data":"6e7b775e3575025f0a3dcf422ab4f326db84798f558deb507cb8bc02f15a5210"} Sep 07 13:07:01.355610 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.355588 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" Sep 07 13:07:01.357400 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.357374 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" event={"ID":"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d","Type":"ContainerStarted","Data":"46d547b0b3ce1c92738705fa8bbd4b83d4d8c4e16d2e0f55161e965d4c5fb711"} Sep 07 13:07:01.357529 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.357424 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" event={"ID":"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d","Type":"ContainerStarted","Data":"03f6d752585cc814da8494f598092dc58046115d309e689a75ae696aba7dfa8c"} Sep 07 13:07:01.357529 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.357477 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" Sep 07 13:07:01.373179 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.373123 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" podStartSLOduration=1.373105013 podStartE2EDuration="1.373105013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:07:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:07:01.372740829 +0000 UTC m=+855.403490629" watchObservedRunningTime="2026-09-07 13:07:01.373105013 +0000 UTC m=+855.403854784" Sep 07 13:07:01.393212 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.393156 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" podStartSLOduration=1.393136102 podStartE2EDuration="1.393136102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:07:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:07:01.390928591 +0000 UTC m=+855.421678362" watchObservedRunningTime="2026-09-07 13:07:01.393136102 +0000 UTC m=+855.423885874" Sep 07 13:07:01.913143 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.913103 2576 patch_prober.go:28] interesting pod/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Sep 07 13:07:01.913631 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:01.913176 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Sep 07 13:07:02.588998 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:02.585732 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:02.722749 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:02.722704 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:02.912504 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:02.912410 2576 patch_prober.go:28] interesting pod/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Sep 07 13:07:02.912504 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:02.912479 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Sep 07 13:07:03.283177 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:03.283145 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:03.399362 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:03.399333 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:03.543500 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:03.543396 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:03.656540 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:03.656506 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:03.774126 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:03.774091 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:03.916084 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:03.916002 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:07:03.916397 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:03.916380 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:07:03.917127 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:03.917108 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:07:04.726537 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:04.726507 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:05.666896 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:05.666866 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:09.389608 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:09.389579 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:10.594291 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:10.594254 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:10.695090 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:10.695053 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:12.364538 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:12.364510 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" Sep 07 13:07:13.372183 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:13.372147 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" Sep 07 13:07:13.832764 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:13.832728 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:18.977126 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:18.977080 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:19.708484 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:19.708449 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:19.789904 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:19.789865 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:19.865878 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:19.865843 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:19.944884 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:19.944848 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:20.064658 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:20.064619 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:20.160440 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:20.160404 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7"] Sep 07 13:07:20.160640 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:20.160617 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerName="istio-proxy" containerID="cri-o://f574aaa9e350ae30961e192b5342197f86c106d3d5d36342a73a0939bcf5c6ae" gracePeriod=30 Sep 07 13:07:20.212167 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:20.212127 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:20.631168 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:20.631134 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:20.710483 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:20.710437 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:20.795198 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:20.795165 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:20.895645 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:20.895548 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:21.219365 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:21.219258 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:21.524562 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:21.524470 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:23.404057 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:23.404022 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:24.160191 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:24.160151 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:24.262946 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:24.262899 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:24.476206 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:24.476107 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:24.569997 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:24.569940 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:25.017169 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.017128 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:25.116892 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.116860 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:25.139180 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.139149 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl"] Sep 07 13:07:25.143014 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.142997 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" Sep 07 13:07:25.146763 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.146737 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-c8f5c2a5\"" Sep 07 13:07:25.155583 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.155563 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl"] Sep 07 13:07:25.263335 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.263295 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48spb\" (UniqueName: \"kubernetes.io/projected/7418736f-382c-44e0-8cab-16be32769cc7-kube-api-access-48spb\") pod \"payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl\" (UID: \"7418736f-382c-44e0-8cab-16be32769cc7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" Sep 07 13:07:25.263335 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.263343 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7418736f-382c-44e0-8cab-16be32769cc7-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl\" (UID: \"7418736f-382c-44e0-8cab-16be32769cc7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" Sep 07 13:07:25.337049 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.337016 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:25.364700 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.364668 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48spb\" (UniqueName: \"kubernetes.io/projected/7418736f-382c-44e0-8cab-16be32769cc7-kube-api-access-48spb\") pod \"payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl\" (UID: \"7418736f-382c-44e0-8cab-16be32769cc7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" Sep 07 13:07:25.364700 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.364703 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7418736f-382c-44e0-8cab-16be32769cc7-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl\" (UID: \"7418736f-382c-44e0-8cab-16be32769cc7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" Sep 07 13:07:25.365376 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.365349 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7418736f-382c-44e0-8cab-16be32769cc7-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl\" (UID: \"7418736f-382c-44e0-8cab-16be32769cc7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" Sep 07 13:07:25.376690 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.376660 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48spb\" (UniqueName: \"kubernetes.io/projected/7418736f-382c-44e0-8cab-16be32769cc7-kube-api-access-48spb\") pod \"payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl\" (UID: \"7418736f-382c-44e0-8cab-16be32769cc7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" Sep 07 13:07:25.438288 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.438263 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:07:25.454165 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.454137 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" Sep 07 13:07:25.488118 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.488081 2576 generic.go:356] "Generic (PLEG): container finished" podID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerID="f574aaa9e350ae30961e192b5342197f86c106d3d5d36342a73a0939bcf5c6ae" exitCode=0 Sep 07 13:07:25.488526 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.488133 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" Sep 07 13:07:25.488526 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.488161 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" event={"ID":"01a7622f-e712-45b4-a6aa-74e827350a4c","Type":"ContainerDied","Data":"f574aaa9e350ae30961e192b5342197f86c106d3d5d36342a73a0939bcf5c6ae"} Sep 07 13:07:25.488526 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.488232 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7" event={"ID":"01a7622f-e712-45b4-a6aa-74e827350a4c","Type":"ContainerDied","Data":"e1d47c8a29d169ae6d30ba14fffab98eb6a7adf3327910cf7bc88142dd10fc03"} Sep 07 13:07:25.488526 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.488264 2576 scope.go:117] "RemoveContainer" containerID="f574aaa9e350ae30961e192b5342197f86c106d3d5d36342a73a0939bcf5c6ae" Sep 07 13:07:25.498259 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.498237 2576 scope.go:117] "RemoveContainer" containerID="6ca1ab146a5d53de87f634c17069d668779bbc24c0127cfc772a374999d17520" Sep 07 13:07:25.508055 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.508020 2576 scope.go:117] "RemoveContainer" containerID="f574aaa9e350ae30961e192b5342197f86c106d3d5d36342a73a0939bcf5c6ae" Sep 07 13:07:25.508369 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:07:25.508333 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f574aaa9e350ae30961e192b5342197f86c106d3d5d36342a73a0939bcf5c6ae\": container with ID starting with f574aaa9e350ae30961e192b5342197f86c106d3d5d36342a73a0939bcf5c6ae not found: ID does not exist" containerID="f574aaa9e350ae30961e192b5342197f86c106d3d5d36342a73a0939bcf5c6ae" Sep 07 13:07:25.508429 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.508379 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f574aaa9e350ae30961e192b5342197f86c106d3d5d36342a73a0939bcf5c6ae"} err="failed to get container status \"f574aaa9e350ae30961e192b5342197f86c106d3d5d36342a73a0939bcf5c6ae\": rpc error: code = NotFound desc = could not find container \"f574aaa9e350ae30961e192b5342197f86c106d3d5d36342a73a0939bcf5c6ae\": container with ID starting with f574aaa9e350ae30961e192b5342197f86c106d3d5d36342a73a0939bcf5c6ae not found: ID does not exist" Sep 07 13:07:25.508429 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.508400 2576 scope.go:117] "RemoveContainer" containerID="6ca1ab146a5d53de87f634c17069d668779bbc24c0127cfc772a374999d17520" Sep 07 13:07:25.508674 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:07:25.508653 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ca1ab146a5d53de87f634c17069d668779bbc24c0127cfc772a374999d17520\": container with ID starting with 6ca1ab146a5d53de87f634c17069d668779bbc24c0127cfc772a374999d17520 not found: ID does not exist" containerID="6ca1ab146a5d53de87f634c17069d668779bbc24c0127cfc772a374999d17520" Sep 07 13:07:25.508770 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.508677 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ca1ab146a5d53de87f634c17069d668779bbc24c0127cfc772a374999d17520"} err="failed to get container status \"6ca1ab146a5d53de87f634c17069d668779bbc24c0127cfc772a374999d17520\": rpc error: code = NotFound desc = could not find container \"6ca1ab146a5d53de87f634c17069d668779bbc24c0127cfc772a374999d17520\": container with ID starting with 6ca1ab146a5d53de87f634c17069d668779bbc24c0127cfc772a374999d17520 not found: ID does not exist" Sep 07 13:07:25.566502 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.566415 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-workload-socket\") pod \"01a7622f-e712-45b4-a6aa-74e827350a4c\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " Sep 07 13:07:25.566502 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.566455 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-token\") pod \"01a7622f-e712-45b4-a6aa-74e827350a4c\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " Sep 07 13:07:25.566502 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.566490 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-envoy\") pod \"01a7622f-e712-45b4-a6aa-74e827350a4c\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " Sep 07 13:07:25.566759 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.566537 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cxgv\" (UniqueName: \"kubernetes.io/projected/01a7622f-e712-45b4-a6aa-74e827350a4c-kube-api-access-5cxgv\") pod \"01a7622f-e712-45b4-a6aa-74e827350a4c\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " Sep 07 13:07:25.566759 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.566560 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-credential-socket\") pod \"01a7622f-e712-45b4-a6aa-74e827350a4c\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " Sep 07 13:07:25.566759 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.566611 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-podinfo\") pod \"01a7622f-e712-45b4-a6aa-74e827350a4c\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " Sep 07 13:07:25.566759 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.566732 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "01a7622f-e712-45b4-a6aa-74e827350a4c" (UID: "01a7622f-e712-45b4-a6aa-74e827350a4c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:07:25.567003 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.566790 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a7622f-e712-45b4-a6aa-74e827350a4c-istiod-ca-cert\") pod \"01a7622f-e712-45b4-a6aa-74e827350a4c\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " Sep 07 13:07:25.567003 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.566880 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-workload-certs\") pod \"01a7622f-e712-45b4-a6aa-74e827350a4c\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " Sep 07 13:07:25.567003 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.566906 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "01a7622f-e712-45b4-a6aa-74e827350a4c" (UID: "01a7622f-e712-45b4-a6aa-74e827350a4c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:07:25.567003 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.566939 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-data\") pod \"01a7622f-e712-45b4-a6aa-74e827350a4c\" (UID: \"01a7622f-e712-45b4-a6aa-74e827350a4c\") " Sep 07 13:07:25.567551 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.567171 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "01a7622f-e712-45b4-a6aa-74e827350a4c" (UID: "01a7622f-e712-45b4-a6aa-74e827350a4c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:07:25.567551 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.567331 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-credential-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:07:25.567551 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.567349 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-workload-certs\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:07:25.567551 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.567363 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-workload-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:07:25.567760 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.567549 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-data" (OuterVolumeSpecName: "istio-data") pod "01a7622f-e712-45b4-a6aa-74e827350a4c" (UID: "01a7622f-e712-45b4-a6aa-74e827350a4c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:07:25.567760 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.567708 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01a7622f-e712-45b4-a6aa-74e827350a4c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "01a7622f-e712-45b4-a6aa-74e827350a4c" (UID: "01a7622f-e712-45b4-a6aa-74e827350a4c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:07:25.569007 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.568962 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "01a7622f-e712-45b4-a6aa-74e827350a4c" (UID: "01a7622f-e712-45b4-a6aa-74e827350a4c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:07:25.569309 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.569278 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-token" (OuterVolumeSpecName: "istio-token") pod "01a7622f-e712-45b4-a6aa-74e827350a4c" (UID: "01a7622f-e712-45b4-a6aa-74e827350a4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:07:25.569395 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.569276 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "01a7622f-e712-45b4-a6aa-74e827350a4c" (UID: "01a7622f-e712-45b4-a6aa-74e827350a4c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 13:07:25.569395 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.569355 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a7622f-e712-45b4-a6aa-74e827350a4c-kube-api-access-5cxgv" (OuterVolumeSpecName: "kube-api-access-5cxgv") pod "01a7622f-e712-45b4-a6aa-74e827350a4c" (UID: "01a7622f-e712-45b4-a6aa-74e827350a4c"). InnerVolumeSpecName "kube-api-access-5cxgv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:07:25.668228 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.668195 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a7622f-e712-45b4-a6aa-74e827350a4c-istiod-ca-cert\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:07:25.668228 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.668230 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-data\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:07:25.668469 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.668242 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-token\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:07:25.668469 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.668295 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-envoy\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:07:25.668469 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.668334 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cxgv\" (UniqueName: \"kubernetes.io/projected/01a7622f-e712-45b4-a6aa-74e827350a4c-kube-api-access-5cxgv\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:07:25.668469 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.668350 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a7622f-e712-45b4-a6aa-74e827350a4c-istio-podinfo\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:07:25.738508 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.738424 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl"] Sep 07 13:07:25.740956 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:07:25.740930 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7418736f_382c_44e0_8cab_16be32769cc7.slice/crio-2d340cc1e27824fc15607861eac87f560c47f43ab6144e0a62dc8f648cb3881f WatchSource:0}: Error finding container 2d340cc1e27824fc15607861eac87f560c47f43ab6144e0a62dc8f648cb3881f: Status 404 returned error can't find the container with id 2d340cc1e27824fc15607861eac87f560c47f43ab6144e0a62dc8f648cb3881f Sep 07 13:07:25.811961 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.811933 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7"] Sep 07 13:07:25.816103 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:25.816074 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2cd995c1-openshift-default-5b78b8cc68-hmrm7"] Sep 07 13:07:26.483433 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:26.483351 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" path="/var/lib/kubelet/pods/01a7622f-e712-45b4-a6aa-74e827350a4c/volumes" Sep 07 13:07:26.494401 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:26.494368 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" event={"ID":"7418736f-382c-44e0-8cab-16be32769cc7","Type":"ContainerStarted","Data":"7b0920e007fd6304636eae2f7402f3e82e6f082ec3e502ff3214eb00f6c9ca9a"} Sep 07 13:07:26.494854 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:26.494420 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" event={"ID":"7418736f-382c-44e0-8cab-16be32769cc7","Type":"ContainerStarted","Data":"2d340cc1e27824fc15607861eac87f560c47f43ab6144e0a62dc8f648cb3881f"} Sep 07 13:07:26.494854 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:26.494450 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" Sep 07 13:07:26.513629 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:26.513574 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" podStartSLOduration=1.513555084 podStartE2EDuration="1.513555084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:07:26.512633844 +0000 UTC m=+880.543383614" watchObservedRunningTime="2026-09-07 13:07:26.513555084 +0000 UTC m=+880.544304856" Sep 07 13:07:37.503080 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:37.503048 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" Sep 07 13:07:42.646116 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:42.646073 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:46.446135 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:46.446107 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 13:07:46.448140 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:46.448119 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 13:07:51.107760 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:51.107724 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:55.713123 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:55.713083 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:55.758516 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:55.758487 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl"] Sep 07 13:07:55.758774 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:55.758741 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" podUID="7418736f-382c-44e0-8cab-16be32769cc7" containerName="payload-pre-processing" containerID="cri-o://7b0920e007fd6304636eae2f7402f3e82e6f082ec3e502ff3214eb00f6c9ca9a" gracePeriod=30 Sep 07 13:07:55.789435 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:55.789403 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:55.856343 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:55.856309 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:55.955794 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:55.955758 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:56.015491 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.015469 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" Sep 07 13:07:56.019377 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.019350 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:56.107754 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.107715 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:56.142618 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.142588 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48spb\" (UniqueName: \"kubernetes.io/projected/7418736f-382c-44e0-8cab-16be32769cc7-kube-api-access-48spb\") pod \"7418736f-382c-44e0-8cab-16be32769cc7\" (UID: \"7418736f-382c-44e0-8cab-16be32769cc7\") " Sep 07 13:07:56.142767 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.142671 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7418736f-382c-44e0-8cab-16be32769cc7-plugins-config-volume\") pod \"7418736f-382c-44e0-8cab-16be32769cc7\" (UID: \"7418736f-382c-44e0-8cab-16be32769cc7\") " Sep 07 13:07:56.143083 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.143051 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7418736f-382c-44e0-8cab-16be32769cc7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7418736f-382c-44e0-8cab-16be32769cc7" (UID: "7418736f-382c-44e0-8cab-16be32769cc7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:07:56.144824 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.144806 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7418736f-382c-44e0-8cab-16be32769cc7-kube-api-access-48spb" (OuterVolumeSpecName: "kube-api-access-48spb") pod "7418736f-382c-44e0-8cab-16be32769cc7" (UID: "7418736f-382c-44e0-8cab-16be32769cc7"). InnerVolumeSpecName "kube-api-access-48spb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:07:56.243381 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.243348 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48spb\" (UniqueName: \"kubernetes.io/projected/7418736f-382c-44e0-8cab-16be32769cc7-kube-api-access-48spb\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:07:56.243381 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.243373 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7418736f-382c-44e0-8cab-16be32769cc7-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:07:56.575855 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.575805 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:56.648618 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.648586 2576 generic.go:356] "Generic (PLEG): container finished" podID="7418736f-382c-44e0-8cab-16be32769cc7" containerID="7b0920e007fd6304636eae2f7402f3e82e6f082ec3e502ff3214eb00f6c9ca9a" exitCode=0 Sep 07 13:07:56.648860 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.648639 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" Sep 07 13:07:56.648860 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.648668 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" event={"ID":"7418736f-382c-44e0-8cab-16be32769cc7","Type":"ContainerDied","Data":"7b0920e007fd6304636eae2f7402f3e82e6f082ec3e502ff3214eb00f6c9ca9a"} Sep 07 13:07:56.648860 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.648721 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl" event={"ID":"7418736f-382c-44e0-8cab-16be32769cc7","Type":"ContainerDied","Data":"2d340cc1e27824fc15607861eac87f560c47f43ab6144e0a62dc8f648cb3881f"} Sep 07 13:07:56.648860 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.648742 2576 scope.go:117] "RemoveContainer" containerID="7b0920e007fd6304636eae2f7402f3e82e6f082ec3e502ff3214eb00f6c9ca9a" Sep 07 13:07:56.657904 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.657881 2576 scope.go:117] "RemoveContainer" containerID="7b0920e007fd6304636eae2f7402f3e82e6f082ec3e502ff3214eb00f6c9ca9a" Sep 07 13:07:56.658216 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:07:56.658198 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b0920e007fd6304636eae2f7402f3e82e6f082ec3e502ff3214eb00f6c9ca9a\": container with ID starting with 7b0920e007fd6304636eae2f7402f3e82e6f082ec3e502ff3214eb00f6c9ca9a not found: ID does not exist" containerID="7b0920e007fd6304636eae2f7402f3e82e6f082ec3e502ff3214eb00f6c9ca9a" Sep 07 13:07:56.658284 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.658225 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b0920e007fd6304636eae2f7402f3e82e6f082ec3e502ff3214eb00f6c9ca9a"} err="failed to get container status \"7b0920e007fd6304636eae2f7402f3e82e6f082ec3e502ff3214eb00f6c9ca9a\": rpc error: code = NotFound desc = could not find container \"7b0920e007fd6304636eae2f7402f3e82e6f082ec3e502ff3214eb00f6c9ca9a\": container with ID starting with 7b0920e007fd6304636eae2f7402f3e82e6f082ec3e502ff3214eb00f6c9ca9a not found: ID does not exist" Sep 07 13:07:56.665371 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.665349 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl"] Sep 07 13:07:56.670710 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.670688 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c8f5c2a5-5989bbf766-vgnxl"] Sep 07 13:07:56.866262 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.866236 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm"] Sep 07 13:07:56.866614 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.866604 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerName="istio-proxy" Sep 07 13:07:56.866655 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.866615 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerName="istio-proxy" Sep 07 13:07:56.866655 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.866633 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerName="istio-proxy" Sep 07 13:07:56.866655 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.866638 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerName="istio-proxy" Sep 07 13:07:56.866655 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.866644 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7418736f-382c-44e0-8cab-16be32769cc7" containerName="payload-pre-processing" Sep 07 13:07:56.866655 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.866649 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7418736f-382c-44e0-8cab-16be32769cc7" containerName="payload-pre-processing" Sep 07 13:07:56.866811 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.866710 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerName="istio-proxy" Sep 07 13:07:56.866811 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.866717 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="01a7622f-e712-45b4-a6aa-74e827350a4c" containerName="istio-proxy" Sep 07 13:07:56.866811 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.866726 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7418736f-382c-44e0-8cab-16be32769cc7" containerName="payload-pre-processing" Sep 07 13:07:56.870915 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.870897 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:56.873499 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.873481 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-40becd28-openshift-default-dockercfg-vpfc2\"" Sep 07 13:07:56.885393 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.885373 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm"] Sep 07 13:07:56.912403 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.912334 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:56.947540 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.947512 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-token\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:56.947676 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.947546 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-podinfo\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:56.947676 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.947579 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-workload-socket\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:56.947676 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.947655 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-data\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:56.947789 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.947690 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnnml\" (UniqueName: \"kubernetes.io/projected/28f17823-b42b-49b6-b18a-9b4b246b9aa8-kube-api-access-qnnml\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:56.947789 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.947748 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-envoy\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:56.947789 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.947764 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istiod-ca-cert\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:56.947888 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.947795 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-credential-socket\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:56.947888 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:56.947818 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-workload-certs\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.048541 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.048508 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-workload-socket\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.048719 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.048557 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-data\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.048719 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.048586 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnnml\" (UniqueName: \"kubernetes.io/projected/28f17823-b42b-49b6-b18a-9b4b246b9aa8-kube-api-access-qnnml\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.048719 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.048626 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-envoy\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.048719 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.048648 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istiod-ca-cert\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.048719 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.048689 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-credential-socket\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.049004 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.048813 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-workload-certs\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.049004 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.048886 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-token\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.049004 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.048915 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-podinfo\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.049004 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.048950 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-workload-socket\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.049004 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.048998 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-data\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.049249 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.049064 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-credential-socket\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.049292 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.049264 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-workload-certs\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.049429 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.049413 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istiod-ca-cert\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.051108 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.051092 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-envoy\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.051483 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.051467 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-podinfo\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.070819 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.070793 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-token\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.071486 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.071456 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnnml\" (UniqueName: \"kubernetes.io/projected/28f17823-b42b-49b6-b18a-9b4b246b9aa8-kube-api-access-qnnml\") pod \"e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.181225 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.181148 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:57.464397 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.464302 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm"] Sep 07 13:07:57.468232 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:07:57.468207 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f17823_b42b_49b6_b18a_9b4b246b9aa8.slice/crio-398dcf530296e05c04a7becd934f2742469a3c6e3b706079477c870f9af1f60a WatchSource:0}: Error finding container 398dcf530296e05c04a7becd934f2742469a3c6e3b706079477c870f9af1f60a: Status 404 returned error can't find the container with id 398dcf530296e05c04a7becd934f2742469a3c6e3b706079477c870f9af1f60a Sep 07 13:07:57.470538 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.470494 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:07:57.470635 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.470585 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:07:57.470635 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.470615 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:07:57.501754 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.501729 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:57.658152 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.658112 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" event={"ID":"28f17823-b42b-49b6-b18a-9b4b246b9aa8","Type":"ContainerStarted","Data":"820541c4547301cd849ee934c115bc9558cb316ff4052cf1f865fa727ca7a329"} Sep 07 13:07:57.658308 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.658171 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" event={"ID":"28f17823-b42b-49b6-b18a-9b4b246b9aa8","Type":"ContainerStarted","Data":"398dcf530296e05c04a7becd934f2742469a3c6e3b706079477c870f9af1f60a"} Sep 07 13:07:57.696389 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:57.696340 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" podStartSLOduration=1.696324919 podStartE2EDuration="1.696324919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:07:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:07:57.692657005 +0000 UTC m=+911.723406775" watchObservedRunningTime="2026-09-07 13:07:57.696324919 +0000 UTC m=+911.727074690" Sep 07 13:07:58.181909 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:58.181877 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:58.186544 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:58.186520 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:58.489141 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:58.489062 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7418736f-382c-44e0-8cab-16be32769cc7" path="/var/lib/kubelet/pods/7418736f-382c-44e0-8cab-16be32769cc7/volumes" Sep 07 13:07:58.663038 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:58.663008 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:58.664018 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:58.663994 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:07:59.078459 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:59.078425 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:07:59.199722 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:07:59.199687 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:00.243868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:00.243832 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:00.344617 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:00.344580 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:00.630010 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:00.629951 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:00.869241 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:00.869209 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:00.967918 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:00.967828 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:01.209628 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:01.209595 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv"] Sep 07 13:08:01.213264 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:01.213247 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" Sep 07 13:08:01.215746 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:01.215700 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-40becd28\"" Sep 07 13:08:01.215843 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:01.215730 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-40becd28-dockercfg-cszmw\"" Sep 07 13:08:01.233501 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:01.233477 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv"] Sep 07 13:08:01.286859 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:01.286822 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hx8c\" (UniqueName: \"kubernetes.io/projected/74f87a16-f394-4d9e-a15a-a38021827f46-kube-api-access-5hx8c\") pod \"payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv\" (UID: \"74f87a16-f394-4d9e-a15a-a38021827f46\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" Sep 07 13:08:01.287246 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:01.286913 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74f87a16-f394-4d9e-a15a-a38021827f46-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv\" (UID: \"74f87a16-f394-4d9e-a15a-a38021827f46\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" Sep 07 13:08:01.387735 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:01.387700 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hx8c\" (UniqueName: \"kubernetes.io/projected/74f87a16-f394-4d9e-a15a-a38021827f46-kube-api-access-5hx8c\") pod \"payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv\" (UID: \"74f87a16-f394-4d9e-a15a-a38021827f46\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" Sep 07 13:08:01.387904 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:01.387772 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74f87a16-f394-4d9e-a15a-a38021827f46-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv\" (UID: \"74f87a16-f394-4d9e-a15a-a38021827f46\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" Sep 07 13:08:01.388422 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:01.388403 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74f87a16-f394-4d9e-a15a-a38021827f46-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv\" (UID: \"74f87a16-f394-4d9e-a15a-a38021827f46\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" Sep 07 13:08:01.398569 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:01.398543 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hx8c\" (UniqueName: \"kubernetes.io/projected/74f87a16-f394-4d9e-a15a-a38021827f46-kube-api-access-5hx8c\") pod \"payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv\" (UID: \"74f87a16-f394-4d9e-a15a-a38021827f46\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" Sep 07 13:08:01.524048 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:01.523930 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" Sep 07 13:08:01.810693 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:01.810667 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv"] Sep 07 13:08:01.812822 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:08:01.812797 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74f87a16_f394_4d9e_a15a_a38021827f46.slice/crio-3683596c346748399375203a256c231cb1f490b63b1631455b620ace5c910c48 WatchSource:0}: Error finding container 3683596c346748399375203a256c231cb1f490b63b1631455b620ace5c910c48: Status 404 returned error can't find the container with id 3683596c346748399375203a256c231cb1f490b63b1631455b620ace5c910c48 Sep 07 13:08:02.685138 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:02.685102 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" event={"ID":"74f87a16-f394-4d9e-a15a-a38021827f46","Type":"ContainerStarted","Data":"fae1b11dae0a39769c26729f09c90514c1a22ac0130cde22b5f7575acfa0088e"} Sep 07 13:08:02.685610 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:02.685258 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" Sep 07 13:08:02.685610 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:02.685284 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" event={"ID":"74f87a16-f394-4d9e-a15a-a38021827f46","Type":"ContainerStarted","Data":"3683596c346748399375203a256c231cb1f490b63b1631455b620ace5c910c48"} Sep 07 13:08:02.703926 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:02.703871 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" podStartSLOduration=1.703851885 podStartE2EDuration="1.703851885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:08:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:08:02.703002712 +0000 UTC m=+916.733752486" watchObservedRunningTime="2026-09-07 13:08:02.703851885 +0000 UTC m=+916.734601656" Sep 07 13:08:13.693509 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:13.693469 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" Sep 07 13:08:21.781985 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:21.781946 2576 generic.go:356] "Generic (PLEG): container finished" podID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerID="820541c4547301cd849ee934c115bc9558cb316ff4052cf1f865fa727ca7a329" exitCode=0 Sep 07 13:08:21.782393 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:21.782026 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" event={"ID":"28f17823-b42b-49b6-b18a-9b4b246b9aa8","Type":"ContainerDied","Data":"820541c4547301cd849ee934c115bc9558cb316ff4052cf1f865fa727ca7a329"} Sep 07 13:08:21.782453 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:21.782427 2576 scope.go:117] "RemoveContainer" containerID="820541c4547301cd849ee934c115bc9558cb316ff4052cf1f865fa727ca7a329" Sep 07 13:08:21.783551 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:21.783524 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:08:21.783654 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:21.783587 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:08:21.783654 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:21.783615 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:08:22.550384 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:22.550353 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:22.793275 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:22.793236 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" event={"ID":"28f17823-b42b-49b6-b18a-9b4b246b9aa8","Type":"ContainerStarted","Data":"b09bce0070a336204ba2c94130f2f5f30fb2c6d8d828cc29f44b07f801fcbaf9"} Sep 07 13:08:23.182271 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:23.182227 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:08:23.182488 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:23.182369 2576 patch_prober.go:28] interesting pod/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Sep 07 13:08:23.182488 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:23.182412 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Sep 07 13:08:23.820389 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:23.820349 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:24.182728 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:24.182627 2576 patch_prober.go:28] interesting pod/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Sep 07 13:08:24.183455 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:24.182996 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Sep 07 13:08:25.003766 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.003729 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq"] Sep 07 13:08:25.014251 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.014218 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq" Sep 07 13:08:25.016065 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.016046 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-flngs\"" Sep 07 13:08:25.016202 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.016053 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 13:08:25.024079 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.024056 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq"] Sep 07 13:08:25.104946 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.104913 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92339f-6a38-4b9d-b2be-5bb81adccafb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-40becd28-7zpbq\" (UID: \"4a92339f-6a38-4b9d-b2be-5bb81adccafb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq" Sep 07 13:08:25.182327 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.182288 2576 patch_prober.go:28] interesting pod/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Sep 07 13:08:25.182614 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.182580 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Sep 07 13:08:25.205721 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.205692 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92339f-6a38-4b9d-b2be-5bb81adccafb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-40becd28-7zpbq\" (UID: \"4a92339f-6a38-4b9d-b2be-5bb81adccafb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq" Sep 07 13:08:25.206331 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.206312 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92339f-6a38-4b9d-b2be-5bb81adccafb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-40becd28-7zpbq\" (UID: \"4a92339f-6a38-4b9d-b2be-5bb81adccafb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq" Sep 07 13:08:25.326243 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.326212 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq" Sep 07 13:08:25.531045 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.531008 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:25.628291 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.628210 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq"] Sep 07 13:08:25.631403 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:08:25.631365 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a92339f_6a38_4b9d_b2be_5bb81adccafb.slice/crio-4c8354799f3f5050b57532062d261d745bb2bdd30708b1a7ec2c943a1548aa8e WatchSource:0}: Error finding container 4c8354799f3f5050b57532062d261d745bb2bdd30708b1a7ec2c943a1548aa8e: Status 404 returned error can't find the container with id 4c8354799f3f5050b57532062d261d745bb2bdd30708b1a7ec2c943a1548aa8e Sep 07 13:08:25.809158 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:25.809123 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq" event={"ID":"4a92339f-6a38-4b9d-b2be-5bb81adccafb","Type":"ContainerStarted","Data":"4c8354799f3f5050b57532062d261d745bb2bdd30708b1a7ec2c943a1548aa8e"} Sep 07 13:08:26.182062 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:26.182021 2576 patch_prober.go:28] interesting pod/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Sep 07 13:08:26.182619 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:26.182587 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Sep 07 13:08:27.183444 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:27.182448 2576 patch_prober.go:28] interesting pod/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Sep 07 13:08:27.183444 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:27.182527 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Sep 07 13:08:27.749395 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:27.749359 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:28.186258 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:28.186222 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:08:28.186606 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:28.186509 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:08:28.187335 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:28.187313 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:08:28.687793 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:28.687745 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:28.828965 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:28.828922 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq" event={"ID":"4a92339f-6a38-4b9d-b2be-5bb81adccafb","Type":"ContainerStarted","Data":"3c68a91ab88d063a1dc6409c717321e914927896f2904ec674b3f09b1e1e0d1d"} Sep 07 13:08:28.862090 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:28.862038 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq" podStartSLOduration=2.297189893 podStartE2EDuration="4.862022422s" totalImagesPullingTime="2.564832529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:08:28.854117705 +0000 UTC m=+942.884867476" watchObservedRunningTime="2026-09-07 13:08:28.862022422 +0000 UTC m=+942.892772222" Sep 07 13:08:29.721908 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:29.721873 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:29.841734 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:29.841696 2576 generic.go:356] "Generic (PLEG): container finished" podID="4a92339f-6a38-4b9d-b2be-5bb81adccafb" containerID="3c68a91ab88d063a1dc6409c717321e914927896f2904ec674b3f09b1e1e0d1d" exitCode=0 Sep 07 13:08:29.841919 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:29.841776 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq" event={"ID":"4a92339f-6a38-4b9d-b2be-5bb81adccafb","Type":"ContainerDied","Data":"3c68a91ab88d063a1dc6409c717321e914927896f2904ec674b3f09b1e1e0d1d"} Sep 07 13:08:30.988529 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:30.988505 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq" Sep 07 13:08:31.062923 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:31.062889 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92339f-6a38-4b9d-b2be-5bb81adccafb-maas-api-service-ca\") pod \"4a92339f-6a38-4b9d-b2be-5bb81adccafb\" (UID: \"4a92339f-6a38-4b9d-b2be-5bb81adccafb\") " Sep 07 13:08:31.063255 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:31.063232 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a92339f-6a38-4b9d-b2be-5bb81adccafb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4a92339f-6a38-4b9d-b2be-5bb81adccafb" (UID: "4a92339f-6a38-4b9d-b2be-5bb81adccafb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:08:31.163652 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:31.163568 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92339f-6a38-4b9d-b2be-5bb81adccafb-maas-api-service-ca\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:31.852612 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:31.852584 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq" Sep 07 13:08:31.852789 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:31.852619 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq" event={"ID":"4a92339f-6a38-4b9d-b2be-5bb81adccafb","Type":"ContainerDied","Data":"4c8354799f3f5050b57532062d261d745bb2bdd30708b1a7ec2c943a1548aa8e"} Sep 07 13:08:31.852789 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:31.852662 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c8354799f3f5050b57532062d261d745bb2bdd30708b1a7ec2c943a1548aa8e" Sep 07 13:08:32.142219 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:32.142120 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:32.865328 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:32.865294 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:32.897929 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:32.897892 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds"] Sep 07 13:08:32.898203 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:32.898170 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" podUID="cebf60da-b7cb-48cc-9336-535074a4ccc0" containerName="payload-processing" containerID="cri-o://5eaf366774a1be8baac20044e96a2faa5476e32ec60e9c58d42dbec02f63e6ba" gracePeriod=30 Sep 07 13:08:32.907056 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:32.907021 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s"] Sep 07 13:08:32.907291 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:32.907257 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" podUID="fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4" containerName="payload-pre-processing" containerID="cri-o://d82c2f7da9863103294f2d989d15b26fada20a19a45a43fe899261a6ecc12dd7" gracePeriod=30 Sep 07 13:08:32.955165 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:32.955132 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:33.032778 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.032732 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:33.155514 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.155455 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:33.220658 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.220629 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" Sep 07 13:08:33.223578 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.223557 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" Sep 07 13:08:33.243400 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.243375 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:33.284520 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.284487 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4-plugins-config-volume\") pod \"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4\" (UID: \"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4\") " Sep 07 13:08:33.284714 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.284556 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cebf60da-b7cb-48cc-9336-535074a4ccc0-plugins-config-volume\") pod \"cebf60da-b7cb-48cc-9336-535074a4ccc0\" (UID: \"cebf60da-b7cb-48cc-9336-535074a4ccc0\") " Sep 07 13:08:33.284714 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.284596 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52mqm\" (UniqueName: \"kubernetes.io/projected/cebf60da-b7cb-48cc-9336-535074a4ccc0-kube-api-access-52mqm\") pod \"cebf60da-b7cb-48cc-9336-535074a4ccc0\" (UID: \"cebf60da-b7cb-48cc-9336-535074a4ccc0\") " Sep 07 13:08:33.284714 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.284623 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbhww\" (UniqueName: \"kubernetes.io/projected/fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4-kube-api-access-kbhww\") pod \"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4\" (UID: \"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4\") " Sep 07 13:08:33.284887 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.284861 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cebf60da-b7cb-48cc-9336-535074a4ccc0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cebf60da-b7cb-48cc-9336-535074a4ccc0" (UID: "cebf60da-b7cb-48cc-9336-535074a4ccc0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:08:33.284948 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.284924 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4" (UID: "fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:08:33.286891 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.286872 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4-kube-api-access-kbhww" (OuterVolumeSpecName: "kube-api-access-kbhww") pod "fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4" (UID: "fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4"). InnerVolumeSpecName "kube-api-access-kbhww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:08:33.286948 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.286919 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cebf60da-b7cb-48cc-9336-535074a4ccc0-kube-api-access-52mqm" (OuterVolumeSpecName: "kube-api-access-52mqm") pod "cebf60da-b7cb-48cc-9336-535074a4ccc0" (UID: "cebf60da-b7cb-48cc-9336-535074a4ccc0"). InnerVolumeSpecName "kube-api-access-52mqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:08:33.386041 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.386001 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cebf60da-b7cb-48cc-9336-535074a4ccc0-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:33.386041 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.386030 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52mqm\" (UniqueName: \"kubernetes.io/projected/cebf60da-b7cb-48cc-9336-535074a4ccc0-kube-api-access-52mqm\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:33.386041 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.386041 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbhww\" (UniqueName: \"kubernetes.io/projected/fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4-kube-api-access-kbhww\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:33.386041 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.386049 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:33.571957 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.571917 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:33.864046 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.863925 2576 generic.go:356] "Generic (PLEG): container finished" podID="cebf60da-b7cb-48cc-9336-535074a4ccc0" containerID="5eaf366774a1be8baac20044e96a2faa5476e32ec60e9c58d42dbec02f63e6ba" exitCode=0 Sep 07 13:08:33.864046 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.863989 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" Sep 07 13:08:33.864046 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.864021 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" event={"ID":"cebf60da-b7cb-48cc-9336-535074a4ccc0","Type":"ContainerDied","Data":"5eaf366774a1be8baac20044e96a2faa5476e32ec60e9c58d42dbec02f63e6ba"} Sep 07 13:08:33.864343 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.864083 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds" event={"ID":"cebf60da-b7cb-48cc-9336-535074a4ccc0","Type":"ContainerDied","Data":"4e699b7bd224fe8891e94a4f0c6b77118cf391ca07b5e6195819bbaa4b42f02c"} Sep 07 13:08:33.864343 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.864114 2576 scope.go:117] "RemoveContainer" containerID="5eaf366774a1be8baac20044e96a2faa5476e32ec60e9c58d42dbec02f63e6ba" Sep 07 13:08:33.865401 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.865378 2576 generic.go:356] "Generic (PLEG): container finished" podID="fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4" containerID="d82c2f7da9863103294f2d989d15b26fada20a19a45a43fe899261a6ecc12dd7" exitCode=0 Sep 07 13:08:33.865476 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.865407 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" Sep 07 13:08:33.865476 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.865458 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" event={"ID":"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4","Type":"ContainerDied","Data":"d82c2f7da9863103294f2d989d15b26fada20a19a45a43fe899261a6ecc12dd7"} Sep 07 13:08:33.865554 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.865498 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s" event={"ID":"fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4","Type":"ContainerDied","Data":"ada20e4cdec0d1e4bcd15c02d6bdc77c4320636708b1ac2a77f1ec72ebddb0ea"} Sep 07 13:08:33.874237 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.874218 2576 scope.go:117] "RemoveContainer" containerID="5eaf366774a1be8baac20044e96a2faa5476e32ec60e9c58d42dbec02f63e6ba" Sep 07 13:08:33.874543 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:08:33.874525 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eaf366774a1be8baac20044e96a2faa5476e32ec60e9c58d42dbec02f63e6ba\": container with ID starting with 5eaf366774a1be8baac20044e96a2faa5476e32ec60e9c58d42dbec02f63e6ba not found: ID does not exist" containerID="5eaf366774a1be8baac20044e96a2faa5476e32ec60e9c58d42dbec02f63e6ba" Sep 07 13:08:33.874596 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.874552 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eaf366774a1be8baac20044e96a2faa5476e32ec60e9c58d42dbec02f63e6ba"} err="failed to get container status \"5eaf366774a1be8baac20044e96a2faa5476e32ec60e9c58d42dbec02f63e6ba\": rpc error: code = NotFound desc = could not find container \"5eaf366774a1be8baac20044e96a2faa5476e32ec60e9c58d42dbec02f63e6ba\": container with ID starting with 5eaf366774a1be8baac20044e96a2faa5476e32ec60e9c58d42dbec02f63e6ba not found: ID does not exist" Sep 07 13:08:33.874596 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.874567 2576 scope.go:117] "RemoveContainer" containerID="d82c2f7da9863103294f2d989d15b26fada20a19a45a43fe899261a6ecc12dd7" Sep 07 13:08:33.882673 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.882654 2576 scope.go:117] "RemoveContainer" containerID="d82c2f7da9863103294f2d989d15b26fada20a19a45a43fe899261a6ecc12dd7" Sep 07 13:08:33.882935 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:08:33.882912 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d82c2f7da9863103294f2d989d15b26fada20a19a45a43fe899261a6ecc12dd7\": container with ID starting with d82c2f7da9863103294f2d989d15b26fada20a19a45a43fe899261a6ecc12dd7 not found: ID does not exist" containerID="d82c2f7da9863103294f2d989d15b26fada20a19a45a43fe899261a6ecc12dd7" Sep 07 13:08:33.883069 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.882940 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d82c2f7da9863103294f2d989d15b26fada20a19a45a43fe899261a6ecc12dd7"} err="failed to get container status \"d82c2f7da9863103294f2d989d15b26fada20a19a45a43fe899261a6ecc12dd7\": rpc error: code = NotFound desc = could not find container \"d82c2f7da9863103294f2d989d15b26fada20a19a45a43fe899261a6ecc12dd7\": container with ID starting with d82c2f7da9863103294f2d989d15b26fada20a19a45a43fe899261a6ecc12dd7 not found: ID does not exist" Sep 07 13:08:33.892188 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.892168 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds"] Sep 07 13:08:33.899147 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.899128 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e9d55a-7bc699777f-z8hds"] Sep 07 13:08:33.915745 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.915725 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s"] Sep 07 13:08:33.918765 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:33.918742 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e9d55a-5bc65cd694-xcq8s"] Sep 07 13:08:34.488473 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:34.488438 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cebf60da-b7cb-48cc-9336-535074a4ccc0" path="/var/lib/kubelet/pods/cebf60da-b7cb-48cc-9336-535074a4ccc0/volumes" Sep 07 13:08:34.488890 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:34.488792 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4" path="/var/lib/kubelet/pods/fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4/volumes" Sep 07 13:08:38.654832 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:38.654793 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:40.432033 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:40.432002 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:41.270597 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.270564 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:41.320955 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.320911 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv"] Sep 07 13:08:41.321270 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.321239 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" podUID="74f87a16-f394-4d9e-a15a-a38021827f46" containerName="payload-pre-processing" containerID="cri-o://fae1b11dae0a39769c26729f09c90514c1a22ac0130cde22b5f7575acfa0088e" gracePeriod=30 Sep 07 13:08:41.355137 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.355105 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:41.422087 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.422053 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:41.503706 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.503679 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq"] Sep 07 13:08:41.507023 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.506992 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-40becd28-7zpbq"] Sep 07 13:08:41.508104 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.508080 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:41.577987 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.577950 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:41.578269 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.578249 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" Sep 07 13:08:41.658021 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.657993 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74f87a16-f394-4d9e-a15a-a38021827f46-plugins-config-volume\") pod \"74f87a16-f394-4d9e-a15a-a38021827f46\" (UID: \"74f87a16-f394-4d9e-a15a-a38021827f46\") " Sep 07 13:08:41.658230 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.658067 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hx8c\" (UniqueName: \"kubernetes.io/projected/74f87a16-f394-4d9e-a15a-a38021827f46-kube-api-access-5hx8c\") pod \"74f87a16-f394-4d9e-a15a-a38021827f46\" (UID: \"74f87a16-f394-4d9e-a15a-a38021827f46\") " Sep 07 13:08:41.658461 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.658431 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74f87a16-f394-4d9e-a15a-a38021827f46-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "74f87a16-f394-4d9e-a15a-a38021827f46" (UID: "74f87a16-f394-4d9e-a15a-a38021827f46"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:08:41.660347 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.660320 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74f87a16-f394-4d9e-a15a-a38021827f46-kube-api-access-5hx8c" (OuterVolumeSpecName: "kube-api-access-5hx8c") pod "74f87a16-f394-4d9e-a15a-a38021827f46" (UID: "74f87a16-f394-4d9e-a15a-a38021827f46"). InnerVolumeSpecName "kube-api-access-5hx8c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:08:41.759633 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.759598 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hx8c\" (UniqueName: \"kubernetes.io/projected/74f87a16-f394-4d9e-a15a-a38021827f46-kube-api-access-5hx8c\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:41.759633 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.759630 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74f87a16-f394-4d9e-a15a-a38021827f46-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:41.916030 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.915908 2576 generic.go:356] "Generic (PLEG): container finished" podID="74f87a16-f394-4d9e-a15a-a38021827f46" containerID="fae1b11dae0a39769c26729f09c90514c1a22ac0130cde22b5f7575acfa0088e" exitCode=0 Sep 07 13:08:41.916030 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.915952 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" Sep 07 13:08:41.916030 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.916008 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" event={"ID":"74f87a16-f394-4d9e-a15a-a38021827f46","Type":"ContainerDied","Data":"fae1b11dae0a39769c26729f09c90514c1a22ac0130cde22b5f7575acfa0088e"} Sep 07 13:08:41.916312 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.916077 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv" event={"ID":"74f87a16-f394-4d9e-a15a-a38021827f46","Type":"ContainerDied","Data":"3683596c346748399375203a256c231cb1f490b63b1631455b620ace5c910c48"} Sep 07 13:08:41.916312 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.916107 2576 scope.go:117] "RemoveContainer" containerID="fae1b11dae0a39769c26729f09c90514c1a22ac0130cde22b5f7575acfa0088e" Sep 07 13:08:41.925128 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.925109 2576 scope.go:117] "RemoveContainer" containerID="fae1b11dae0a39769c26729f09c90514c1a22ac0130cde22b5f7575acfa0088e" Sep 07 13:08:41.925420 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:08:41.925398 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fae1b11dae0a39769c26729f09c90514c1a22ac0130cde22b5f7575acfa0088e\": container with ID starting with fae1b11dae0a39769c26729f09c90514c1a22ac0130cde22b5f7575acfa0088e not found: ID does not exist" containerID="fae1b11dae0a39769c26729f09c90514c1a22ac0130cde22b5f7575acfa0088e" Sep 07 13:08:41.925474 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.925435 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fae1b11dae0a39769c26729f09c90514c1a22ac0130cde22b5f7575acfa0088e"} err="failed to get container status \"fae1b11dae0a39769c26729f09c90514c1a22ac0130cde22b5f7575acfa0088e\": rpc error: code = NotFound desc = could not find container \"fae1b11dae0a39769c26729f09c90514c1a22ac0130cde22b5f7575acfa0088e\": container with ID starting with fae1b11dae0a39769c26729f09c90514c1a22ac0130cde22b5f7575acfa0088e not found: ID does not exist" Sep 07 13:08:41.938999 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.938964 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv"] Sep 07 13:08:41.945680 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:41.945663 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-40becd28-5bd8f6b678-6p9kv"] Sep 07 13:08:42.483347 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:42.483318 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a92339f-6a38-4b9d-b2be-5bb81adccafb" path="/var/lib/kubelet/pods/4a92339f-6a38-4b9d-b2be-5bb81adccafb/volumes" Sep 07 13:08:42.483672 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:42.483659 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74f87a16-f394-4d9e-a15a-a38021827f46" path="/var/lib/kubelet/pods/74f87a16-f394-4d9e-a15a-a38021827f46/volumes" Sep 07 13:08:42.709142 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:42.709112 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm"] Sep 07 13:08:42.709547 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:42.709330 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerName="istio-proxy" containerID="cri-o://b09bce0070a336204ba2c94130f2f5f30fb2c6d8d828cc29f44b07f801fcbaf9" gracePeriod=30 Sep 07 13:08:42.724725 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:42.724701 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:43.187125 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.187081 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:43.188515 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.188488 2576 patch_prober.go:28] interesting pod/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 07 13:08:43.188631 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.188537 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 13:08:43.508190 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.508098 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq"] Sep 07 13:08:43.508555 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.508539 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4" containerName="payload-pre-processing" Sep 07 13:08:43.508647 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.508559 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4" containerName="payload-pre-processing" Sep 07 13:08:43.508647 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.508574 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74f87a16-f394-4d9e-a15a-a38021827f46" containerName="payload-pre-processing" Sep 07 13:08:43.508647 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.508583 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="74f87a16-f394-4d9e-a15a-a38021827f46" containerName="payload-pre-processing" Sep 07 13:08:43.508647 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.508608 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a92339f-6a38-4b9d-b2be-5bb81adccafb" containerName="revoke-keys" Sep 07 13:08:43.508647 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.508616 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a92339f-6a38-4b9d-b2be-5bb81adccafb" containerName="revoke-keys" Sep 07 13:08:43.508647 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.508628 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cebf60da-b7cb-48cc-9336-535074a4ccc0" containerName="payload-processing" Sep 07 13:08:43.508647 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.508636 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cebf60da-b7cb-48cc-9336-535074a4ccc0" containerName="payload-processing" Sep 07 13:08:43.509009 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.508754 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd73fb5d-a518-4e41-b7ff-b6a37fceb1a4" containerName="payload-pre-processing" Sep 07 13:08:43.509009 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.508768 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cebf60da-b7cb-48cc-9336-535074a4ccc0" containerName="payload-processing" Sep 07 13:08:43.509009 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.508779 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="74f87a16-f394-4d9e-a15a-a38021827f46" containerName="payload-pre-processing" Sep 07 13:08:43.509009 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.508789 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a92339f-6a38-4b9d-b2be-5bb81adccafb" containerName="revoke-keys" Sep 07 13:08:43.513263 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.513241 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.515261 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.515242 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-42173418-openshift-default-dockercfg-2l96s\"" Sep 07 13:08:43.523409 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.523385 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq"] Sep 07 13:08:43.537092 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.537066 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:43.577862 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.577829 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-credential-socket\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.578023 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.577869 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/931c134a-aecc-410f-ac84-64836b04bd09-istio-podinfo\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.578023 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.577892 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/931c134a-aecc-410f-ac84-64836b04bd09-istio-token\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.578023 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.577951 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-workload-certs\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.578023 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.577996 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/931c134a-aecc-410f-ac84-64836b04bd09-istiod-ca-cert\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.578023 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.578018 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-workload-socket\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.578219 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.578032 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-istio-envoy\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.578219 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.578124 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-istio-data\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.578219 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.578163 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkjth\" (UniqueName: \"kubernetes.io/projected/931c134a-aecc-410f-ac84-64836b04bd09-kube-api-access-dkjth\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.678870 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.678835 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/931c134a-aecc-410f-ac84-64836b04bd09-istio-podinfo\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.679093 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.678883 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/931c134a-aecc-410f-ac84-64836b04bd09-istio-token\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.679093 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.678929 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-workload-certs\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.679093 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.678954 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/931c134a-aecc-410f-ac84-64836b04bd09-istiod-ca-cert\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.679093 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.679002 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-workload-socket\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.679093 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.679024 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-istio-envoy\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.679367 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.679120 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-istio-data\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.679367 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.679230 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkjth\" (UniqueName: \"kubernetes.io/projected/931c134a-aecc-410f-ac84-64836b04bd09-kube-api-access-dkjth\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.679367 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.679289 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-credential-socket\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.680166 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.679863 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-workload-socket\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.680166 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.680123 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-workload-certs\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.680335 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.680180 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/931c134a-aecc-410f-ac84-64836b04bd09-istiod-ca-cert\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.680584 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.680562 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-istio-data\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.680927 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.680895 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-credential-socket\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.682349 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.682331 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/931c134a-aecc-410f-ac84-64836b04bd09-istio-podinfo\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.682579 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.682562 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-istio-envoy\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.691289 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.691262 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkjth\" (UniqueName: \"kubernetes.io/projected/931c134a-aecc-410f-ac84-64836b04bd09-kube-api-access-dkjth\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.692410 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.692364 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/931c134a-aecc-410f-ac84-64836b04bd09-istio-token\") pod \"e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:43.824791 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:43.824756 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:44.108296 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:44.108203 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq"] Sep 07 13:08:44.112033 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:08:44.112002 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod931c134a_aecc_410f_ac84_64836b04bd09.slice/crio-150a6c25a94fed4e55c85ec7445d7ec1ac8d2736ee5960b47953e6e7beaebd9f WatchSource:0}: Error finding container 150a6c25a94fed4e55c85ec7445d7ec1ac8d2736ee5960b47953e6e7beaebd9f: Status 404 returned error can't find the container with id 150a6c25a94fed4e55c85ec7445d7ec1ac8d2736ee5960b47953e6e7beaebd9f Sep 07 13:08:44.114395 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:44.114366 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:08:44.114474 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:44.114433 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:08:44.114474 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:44.114461 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:08:44.158198 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:44.158171 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:44.942252 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:44.942217 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" event={"ID":"931c134a-aecc-410f-ac84-64836b04bd09","Type":"ContainerStarted","Data":"6c13709e6b17a35d0fc032ea12db9c1bfcac3977b62f5e4135451fb623971b99"} Sep 07 13:08:44.942643 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:44.942294 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" event={"ID":"931c134a-aecc-410f-ac84-64836b04bd09","Type":"ContainerStarted","Data":"150a6c25a94fed4e55c85ec7445d7ec1ac8d2736ee5960b47953e6e7beaebd9f"} Sep 07 13:08:44.962070 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:44.962018 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" podStartSLOduration=1.961999065 podStartE2EDuration="1.961999065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:08:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:08:44.959776055 +0000 UTC m=+958.990525837" watchObservedRunningTime="2026-09-07 13:08:44.961999065 +0000 UTC m=+958.992748832" Sep 07 13:08:45.825871 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:45.825830 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:45.831487 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:45.831452 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:45.947771 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:45.947739 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:45.948828 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:45.948808 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:08:46.311669 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:46.311632 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:46.444743 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:46.444706 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:46.540433 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:46.540396 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:47.603453 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:47.603420 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:47.674727 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:47.674691 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:47.775194 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:47.775167 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:47.958633 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:47.958606 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:08:47.959797 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:47.959775 2576 generic.go:356] "Generic (PLEG): container finished" podID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerID="b09bce0070a336204ba2c94130f2f5f30fb2c6d8d828cc29f44b07f801fcbaf9" exitCode=0 Sep 07 13:08:47.959905 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:47.959842 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" event={"ID":"28f17823-b42b-49b6-b18a-9b4b246b9aa8","Type":"ContainerDied","Data":"b09bce0070a336204ba2c94130f2f5f30fb2c6d8d828cc29f44b07f801fcbaf9"} Sep 07 13:08:47.959905 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:47.959897 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" event={"ID":"28f17823-b42b-49b6-b18a-9b4b246b9aa8","Type":"ContainerDied","Data":"398dcf530296e05c04a7becd934f2742469a3c6e3b706079477c870f9af1f60a"} Sep 07 13:08:47.960020 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:47.959919 2576 scope.go:117] "RemoveContainer" containerID="b09bce0070a336204ba2c94130f2f5f30fb2c6d8d828cc29f44b07f801fcbaf9" Sep 07 13:08:47.968862 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:47.968843 2576 scope.go:117] "RemoveContainer" containerID="820541c4547301cd849ee934c115bc9558cb316ff4052cf1f865fa727ca7a329" Sep 07 13:08:47.976992 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:47.976949 2576 scope.go:117] "RemoveContainer" containerID="b09bce0070a336204ba2c94130f2f5f30fb2c6d8d828cc29f44b07f801fcbaf9" Sep 07 13:08:47.977266 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:08:47.977247 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b09bce0070a336204ba2c94130f2f5f30fb2c6d8d828cc29f44b07f801fcbaf9\": container with ID starting with b09bce0070a336204ba2c94130f2f5f30fb2c6d8d828cc29f44b07f801fcbaf9 not found: ID does not exist" containerID="b09bce0070a336204ba2c94130f2f5f30fb2c6d8d828cc29f44b07f801fcbaf9" Sep 07 13:08:47.977329 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:47.977276 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b09bce0070a336204ba2c94130f2f5f30fb2c6d8d828cc29f44b07f801fcbaf9"} err="failed to get container status \"b09bce0070a336204ba2c94130f2f5f30fb2c6d8d828cc29f44b07f801fcbaf9\": rpc error: code = NotFound desc = could not find container \"b09bce0070a336204ba2c94130f2f5f30fb2c6d8d828cc29f44b07f801fcbaf9\": container with ID starting with b09bce0070a336204ba2c94130f2f5f30fb2c6d8d828cc29f44b07f801fcbaf9 not found: ID does not exist" Sep 07 13:08:47.977329 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:47.977300 2576 scope.go:117] "RemoveContainer" containerID="820541c4547301cd849ee934c115bc9558cb316ff4052cf1f865fa727ca7a329" Sep 07 13:08:47.977558 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:08:47.977540 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"820541c4547301cd849ee934c115bc9558cb316ff4052cf1f865fa727ca7a329\": container with ID starting with 820541c4547301cd849ee934c115bc9558cb316ff4052cf1f865fa727ca7a329 not found: ID does not exist" containerID="820541c4547301cd849ee934c115bc9558cb316ff4052cf1f865fa727ca7a329" Sep 07 13:08:47.977609 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:47.977565 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"820541c4547301cd849ee934c115bc9558cb316ff4052cf1f865fa727ca7a329"} err="failed to get container status \"820541c4547301cd849ee934c115bc9558cb316ff4052cf1f865fa727ca7a329\": rpc error: code = NotFound desc = could not find container \"820541c4547301cd849ee934c115bc9558cb316ff4052cf1f865fa727ca7a329\": container with ID starting with 820541c4547301cd849ee934c115bc9558cb316ff4052cf1f865fa727ca7a329 not found: ID does not exist" Sep 07 13:08:48.024954 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.024921 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-envoy\") pod \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " Sep 07 13:08:48.025148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.024965 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-workload-certs\") pod \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " Sep 07 13:08:48.025148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.025000 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-token\") pod \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " Sep 07 13:08:48.025148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.025019 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istiod-ca-cert\") pod \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " Sep 07 13:08:48.025148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.025050 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-data\") pod \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " Sep 07 13:08:48.025148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.025096 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-credential-socket\") pod \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " Sep 07 13:08:48.025401 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.025157 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnnml\" (UniqueName: \"kubernetes.io/projected/28f17823-b42b-49b6-b18a-9b4b246b9aa8-kube-api-access-qnnml\") pod \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " Sep 07 13:08:48.025401 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.025179 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-podinfo\") pod \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " Sep 07 13:08:48.025401 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.025209 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-workload-socket\") pod \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\" (UID: \"28f17823-b42b-49b6-b18a-9b4b246b9aa8\") " Sep 07 13:08:48.025401 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.025290 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "28f17823-b42b-49b6-b18a-9b4b246b9aa8" (UID: "28f17823-b42b-49b6-b18a-9b4b246b9aa8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:08:48.025620 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.025557 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "28f17823-b42b-49b6-b18a-9b4b246b9aa8" (UID: "28f17823-b42b-49b6-b18a-9b4b246b9aa8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:08:48.025620 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.025591 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "28f17823-b42b-49b6-b18a-9b4b246b9aa8" (UID: "28f17823-b42b-49b6-b18a-9b4b246b9aa8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:08:48.025718 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.025643 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-workload-certs\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:48.025718 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.025662 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-credential-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:48.025817 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.025762 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-data" (OuterVolumeSpecName: "istio-data") pod "28f17823-b42b-49b6-b18a-9b4b246b9aa8" (UID: "28f17823-b42b-49b6-b18a-9b4b246b9aa8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:08:48.026069 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.026045 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "28f17823-b42b-49b6-b18a-9b4b246b9aa8" (UID: "28f17823-b42b-49b6-b18a-9b4b246b9aa8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:08:48.027894 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.027872 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "28f17823-b42b-49b6-b18a-9b4b246b9aa8" (UID: "28f17823-b42b-49b6-b18a-9b4b246b9aa8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:08:48.027985 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.027891 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-token" (OuterVolumeSpecName: "istio-token") pod "28f17823-b42b-49b6-b18a-9b4b246b9aa8" (UID: "28f17823-b42b-49b6-b18a-9b4b246b9aa8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:08:48.028048 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.028018 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "28f17823-b42b-49b6-b18a-9b4b246b9aa8" (UID: "28f17823-b42b-49b6-b18a-9b4b246b9aa8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 13:08:48.028048 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.028018 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28f17823-b42b-49b6-b18a-9b4b246b9aa8-kube-api-access-qnnml" (OuterVolumeSpecName: "kube-api-access-qnnml") pod "28f17823-b42b-49b6-b18a-9b4b246b9aa8" (UID: "28f17823-b42b-49b6-b18a-9b4b246b9aa8"). InnerVolumeSpecName "kube-api-access-qnnml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:08:48.127213 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.127130 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-token\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:48.127213 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.127155 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istiod-ca-cert\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:48.127213 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.127165 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-data\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:48.127213 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.127172 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnnml\" (UniqueName: \"kubernetes.io/projected/28f17823-b42b-49b6-b18a-9b4b246b9aa8-kube-api-access-qnnml\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:48.127213 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.127181 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-podinfo\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:48.127213 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.127189 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-workload-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:48.127213 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.127197 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28f17823-b42b-49b6-b18a-9b4b246b9aa8-istio-envoy\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:08:48.360644 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.360611 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp"] Sep 07 13:08:48.360999 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.360986 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerName="istio-proxy" Sep 07 13:08:48.361053 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.361000 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerName="istio-proxy" Sep 07 13:08:48.361053 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.361022 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerName="istio-proxy" Sep 07 13:08:48.361053 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.361027 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerName="istio-proxy" Sep 07 13:08:48.361164 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.361082 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerName="istio-proxy" Sep 07 13:08:48.365381 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.365364 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" Sep 07 13:08:48.369221 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.369203 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-42173418\"" Sep 07 13:08:48.370386 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.370368 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-42173418-dockercfg-zhkdd\"" Sep 07 13:08:48.377922 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.377864 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp"] Sep 07 13:08:48.413249 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.413184 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:48.430750 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.430718 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7b6148b-0e51-479d-839e-9d8589a59075-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp\" (UID: \"d7b6148b-0e51-479d-839e-9d8589a59075\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" Sep 07 13:08:48.430899 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.430802 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7cgd\" (UniqueName: \"kubernetes.io/projected/d7b6148b-0e51-479d-839e-9d8589a59075-kube-api-access-p7cgd\") pod \"payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp\" (UID: \"d7b6148b-0e51-479d-839e-9d8589a59075\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" Sep 07 13:08:48.515218 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.515184 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:08:48.531211 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.531186 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7cgd\" (UniqueName: \"kubernetes.io/projected/d7b6148b-0e51-479d-839e-9d8589a59075-kube-api-access-p7cgd\") pod \"payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp\" (UID: \"d7b6148b-0e51-479d-839e-9d8589a59075\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" Sep 07 13:08:48.531294 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.531246 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7b6148b-0e51-479d-839e-9d8589a59075-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp\" (UID: \"d7b6148b-0e51-479d-839e-9d8589a59075\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" Sep 07 13:08:48.531835 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.531817 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7b6148b-0e51-479d-839e-9d8589a59075-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp\" (UID: \"d7b6148b-0e51-479d-839e-9d8589a59075\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" Sep 07 13:08:48.542293 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.542268 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7cgd\" (UniqueName: \"kubernetes.io/projected/d7b6148b-0e51-479d-839e-9d8589a59075-kube-api-access-p7cgd\") pod \"payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp\" (UID: \"d7b6148b-0e51-479d-839e-9d8589a59075\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" Sep 07 13:08:48.676876 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.676795 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" Sep 07 13:08:48.952483 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.952458 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp"] Sep 07 13:08:48.954444 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:08:48.954417 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7b6148b_0e51_479d_839e_9d8589a59075.slice/crio-48f1d9e55b445e37a747248b353331ffb04641829a17c5b060cee55a7bb88dee WatchSource:0}: Error finding container 48f1d9e55b445e37a747248b353331ffb04641829a17c5b060cee55a7bb88dee: Status 404 returned error can't find the container with id 48f1d9e55b445e37a747248b353331ffb04641829a17c5b060cee55a7bb88dee Sep 07 13:08:48.966662 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.966632 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" event={"ID":"d7b6148b-0e51-479d-839e-9d8589a59075","Type":"ContainerStarted","Data":"48f1d9e55b445e37a747248b353331ffb04641829a17c5b060cee55a7bb88dee"} Sep 07 13:08:48.967589 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.967573 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm" Sep 07 13:08:48.990703 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:48.990684 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm"] Sep 07 13:08:49.000830 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:49.000808 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-40becd28-openshift-default-8cd9c56f6-nrchm"] Sep 07 13:08:49.976500 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:49.976458 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" event={"ID":"d7b6148b-0e51-479d-839e-9d8589a59075","Type":"ContainerStarted","Data":"9dbc014ac88bd9cc3c06f45551c0609bd094b3695bd5a0150d2e21933ee0904a"} Sep 07 13:08:49.976939 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:49.976587 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" Sep 07 13:08:49.994394 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:49.994332 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" podStartSLOduration=1.994309728 podStartE2EDuration="1.994309728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:08:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:08:49.991790313 +0000 UTC m=+964.022540094" watchObservedRunningTime="2026-09-07 13:08:49.994309728 +0000 UTC m=+964.025059503" Sep 07 13:08:50.482898 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:08:50.482865 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" path="/var/lib/kubelet/pods/28f17823-b42b-49b6-b18a-9b4b246b9aa8/volumes" Sep 07 13:09:00.984385 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:00.984330 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" Sep 07 13:09:01.248766 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:01.248680 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:01.645612 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:01.645579 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:01.905638 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:01.905535 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:02.359457 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.359419 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:02.666136 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.666046 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f"] Sep 07 13:09:02.666795 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.666772 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f17823-b42b-49b6-b18a-9b4b246b9aa8" containerName="istio-proxy" Sep 07 13:09:02.695276 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.691575 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f"] Sep 07 13:09:02.695276 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.691705 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.695276 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.695228 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-02ea24-openshift-default-dockercfg-8k7dw\"" Sep 07 13:09:02.698781 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.698755 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:02.764480 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.764442 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-podinfo\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.764480 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.764483 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-token\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.764718 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.764516 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-credential-socket\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.764718 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.764641 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-workload-socket\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.764831 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.764715 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-data\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.764831 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.764746 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw7xg\" (UniqueName: \"kubernetes.io/projected/a325cbe3-b411-4970-889c-91f05a8bb49f-kube-api-access-hw7xg\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.764831 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.764777 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a325cbe3-b411-4970-889c-91f05a8bb49f-istiod-ca-cert\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.765001 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.764870 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-workload-certs\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.765001 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.764954 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-envoy\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.865473 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.865424 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-podinfo\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.865473 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.865479 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-token\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.865750 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.865508 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-credential-socket\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.865750 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.865548 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-workload-socket\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.865750 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.865601 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-data\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.865750 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.865623 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw7xg\" (UniqueName: \"kubernetes.io/projected/a325cbe3-b411-4970-889c-91f05a8bb49f-kube-api-access-hw7xg\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.865750 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.865649 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a325cbe3-b411-4970-889c-91f05a8bb49f-istiod-ca-cert\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.865750 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.865707 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-workload-certs\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.865750 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.865732 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-envoy\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.866147 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.866043 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-credential-socket\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.866206 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.866139 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-data\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.866206 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.866165 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-workload-socket\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.866310 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.866244 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-workload-certs\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.866431 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.866412 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a325cbe3-b411-4970-889c-91f05a8bb49f-istiod-ca-cert\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.868687 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.868622 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-podinfo\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.871191 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.871168 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-envoy\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.884651 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.884592 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-token\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:02.885352 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.885295 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:02.888999 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:02.887877 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw7xg\" (UniqueName: \"kubernetes.io/projected/a325cbe3-b411-4970-889c-91f05a8bb49f-kube-api-access-hw7xg\") pod \"e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:03.010106 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:03.010007 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:03.300696 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:03.300664 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f"] Sep 07 13:09:03.304076 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:09:03.304048 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda325cbe3_b411_4970_889c_91f05a8bb49f.slice/crio-7b713788c685db359c32fc04133d1d1923b271b1ef641523541136babceeebca WatchSource:0}: Error finding container 7b713788c685db359c32fc04133d1d1923b271b1ef641523541136babceeebca: Status 404 returned error can't find the container with id 7b713788c685db359c32fc04133d1d1923b271b1ef641523541136babceeebca Sep 07 13:09:03.306128 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:03.306099 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:09:03.306206 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:03.306162 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:09:03.306206 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:03.306190 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:09:04.049604 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:04.049565 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" event={"ID":"a325cbe3-b411-4970-889c-91f05a8bb49f","Type":"ContainerStarted","Data":"aa84c1e5b7dcdae4a453966baf67eccc7402607e4e16ab1afdbff763919b4bd1"} Sep 07 13:09:04.050007 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:04.049627 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" event={"ID":"a325cbe3-b411-4970-889c-91f05a8bb49f","Type":"ContainerStarted","Data":"7b713788c685db359c32fc04133d1d1923b271b1ef641523541136babceeebca"} Sep 07 13:09:04.077062 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:04.077003 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" podStartSLOduration=2.076961884 podStartE2EDuration="2.076961884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:09:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:09:04.074811489 +0000 UTC m=+978.105561280" watchObservedRunningTime="2026-09-07 13:09:04.076961884 +0000 UTC m=+978.107711657" Sep 07 13:09:05.010562 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:05.010506 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:05.016599 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:05.016569 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:05.054400 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:05.054369 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:05.055350 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:05.055327 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:06.181419 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:06.181388 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:06.381181 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:06.381142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:07.065157 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:07.065127 2576 generic.go:356] "Generic (PLEG): container finished" podID="931c134a-aecc-410f-ac84-64836b04bd09" containerID="6c13709e6b17a35d0fc032ea12db9c1bfcac3977b62f5e4135451fb623971b99" exitCode=0 Sep 07 13:09:07.065361 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:07.065202 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" event={"ID":"931c134a-aecc-410f-ac84-64836b04bd09","Type":"ContainerDied","Data":"6c13709e6b17a35d0fc032ea12db9c1bfcac3977b62f5e4135451fb623971b99"} Sep 07 13:09:07.065727 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:07.065699 2576 scope.go:117] "RemoveContainer" containerID="6c13709e6b17a35d0fc032ea12db9c1bfcac3977b62f5e4135451fb623971b99" Sep 07 13:09:07.066663 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:07.066640 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:09:07.066731 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:07.066702 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:09:07.066774 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:07.066732 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:09:07.409884 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:07.409800 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:07.524087 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:07.524050 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:07.624181 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:07.624101 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:07.910855 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:07.910813 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:08.009105 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:08.009064 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:08.096579 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:08.096546 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" event={"ID":"931c134a-aecc-410f-ac84-64836b04bd09","Type":"ContainerStarted","Data":"fb4c3ec4b1e8e23b6aad7cf34e4f265623742a4898d1e7fbe344386d920a7ae3"} Sep 07 13:09:08.804921 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:08.804885 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:08.825964 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:08.825929 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:09:08.826208 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:08.826189 2576 patch_prober.go:28] interesting pod/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 07 13:09:08.826264 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:08.826234 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" podUID="931c134a-aecc-410f-ac84-64836b04bd09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 07 13:09:09.825407 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:09.825367 2576 patch_prober.go:28] interesting pod/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 07 13:09:09.825789 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:09.825424 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" podUID="931c134a-aecc-410f-ac84-64836b04bd09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 07 13:09:10.826124 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:10.826084 2576 patch_prober.go:28] interesting pod/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 07 13:09:10.826523 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:10.826141 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" podUID="931c134a-aecc-410f-ac84-64836b04bd09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 07 13:09:11.825708 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:11.825671 2576 patch_prober.go:28] interesting pod/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 07 13:09:11.825894 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:11.825731 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" podUID="931c134a-aecc-410f-ac84-64836b04bd09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 07 13:09:12.826125 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:12.826087 2576 patch_prober.go:28] interesting pod/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 07 13:09:12.826514 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:12.826162 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" podUID="931c134a-aecc-410f-ac84-64836b04bd09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 07 13:09:13.829558 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:13.829529 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:09:13.829954 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:13.829807 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:09:13.830490 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:13.830474 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:09:18.193997 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.190530 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:18.217900 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.217864 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp"] Sep 07 13:09:18.218165 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.218136 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" podUID="d7b6148b-0e51-479d-839e-9d8589a59075" containerName="payload-pre-processing" containerID="cri-o://9dbc014ac88bd9cc3c06f45551c0609bd094b3695bd5a0150d2e21933ee0904a" gracePeriod=30 Sep 07 13:09:18.283844 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.283794 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:18.387586 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.387542 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:18.454426 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.454334 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:18.518856 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.518833 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" Sep 07 13:09:18.532570 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.532539 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq"] Sep 07 13:09:18.532889 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.532823 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" podUID="931c134a-aecc-410f-ac84-64836b04bd09" containerName="istio-proxy" containerID="cri-o://fb4c3ec4b1e8e23b6aad7cf34e4f265623742a4898d1e7fbe344386d920a7ae3" gracePeriod=30 Sep 07 13:09:18.573500 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.567106 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:18.626308 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.626273 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7b6148b-0e51-479d-839e-9d8589a59075-plugins-config-volume\") pod \"d7b6148b-0e51-479d-839e-9d8589a59075\" (UID: \"d7b6148b-0e51-479d-839e-9d8589a59075\") " Sep 07 13:09:18.626511 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.626359 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7cgd\" (UniqueName: \"kubernetes.io/projected/d7b6148b-0e51-479d-839e-9d8589a59075-kube-api-access-p7cgd\") pod \"d7b6148b-0e51-479d-839e-9d8589a59075\" (UID: \"d7b6148b-0e51-479d-839e-9d8589a59075\") " Sep 07 13:09:18.627271 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.627237 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7b6148b-0e51-479d-839e-9d8589a59075-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d7b6148b-0e51-479d-839e-9d8589a59075" (UID: "d7b6148b-0e51-479d-839e-9d8589a59075"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:09:18.630211 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.630165 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7b6148b-0e51-479d-839e-9d8589a59075-kube-api-access-p7cgd" (OuterVolumeSpecName: "kube-api-access-p7cgd") pod "d7b6148b-0e51-479d-839e-9d8589a59075" (UID: "d7b6148b-0e51-479d-839e-9d8589a59075"). InnerVolumeSpecName "kube-api-access-p7cgd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:09:18.727588 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.727510 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7b6148b-0e51-479d-839e-9d8589a59075-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:09:18.727588 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:18.727536 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7cgd\" (UniqueName: \"kubernetes.io/projected/d7b6148b-0e51-479d-839e-9d8589a59075-kube-api-access-p7cgd\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:09:19.156761 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:19.156721 2576 generic.go:356] "Generic (PLEG): container finished" podID="d7b6148b-0e51-479d-839e-9d8589a59075" containerID="9dbc014ac88bd9cc3c06f45551c0609bd094b3695bd5a0150d2e21933ee0904a" exitCode=0 Sep 07 13:09:19.156960 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:19.156774 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" Sep 07 13:09:19.156960 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:19.156799 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" event={"ID":"d7b6148b-0e51-479d-839e-9d8589a59075","Type":"ContainerDied","Data":"9dbc014ac88bd9cc3c06f45551c0609bd094b3695bd5a0150d2e21933ee0904a"} Sep 07 13:09:19.156960 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:19.156862 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp" event={"ID":"d7b6148b-0e51-479d-839e-9d8589a59075","Type":"ContainerDied","Data":"48f1d9e55b445e37a747248b353331ffb04641829a17c5b060cee55a7bb88dee"} Sep 07 13:09:19.157648 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:19.156894 2576 scope.go:117] "RemoveContainer" containerID="9dbc014ac88bd9cc3c06f45551c0609bd094b3695bd5a0150d2e21933ee0904a" Sep 07 13:09:19.172311 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:19.172291 2576 scope.go:117] "RemoveContainer" containerID="9dbc014ac88bd9cc3c06f45551c0609bd094b3695bd5a0150d2e21933ee0904a" Sep 07 13:09:19.172660 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:09:19.172630 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dbc014ac88bd9cc3c06f45551c0609bd094b3695bd5a0150d2e21933ee0904a\": container with ID starting with 9dbc014ac88bd9cc3c06f45551c0609bd094b3695bd5a0150d2e21933ee0904a not found: ID does not exist" containerID="9dbc014ac88bd9cc3c06f45551c0609bd094b3695bd5a0150d2e21933ee0904a" Sep 07 13:09:19.172757 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:19.172670 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dbc014ac88bd9cc3c06f45551c0609bd094b3695bd5a0150d2e21933ee0904a"} err="failed to get container status \"9dbc014ac88bd9cc3c06f45551c0609bd094b3695bd5a0150d2e21933ee0904a\": rpc error: code = NotFound desc = could not find container \"9dbc014ac88bd9cc3c06f45551c0609bd094b3695bd5a0150d2e21933ee0904a\": container with ID starting with 9dbc014ac88bd9cc3c06f45551c0609bd094b3695bd5a0150d2e21933ee0904a not found: ID does not exist" Sep 07 13:09:19.185368 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:19.185327 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp"] Sep 07 13:09:19.188567 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:19.188543 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-42173418-7dd67847bb-fn2qp"] Sep 07 13:09:19.338327 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:19.338286 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:19.431316 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:19.431213 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:19.531413 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:19.531369 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:20.483725 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:20.483682 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7b6148b-0e51-479d-839e-9d8589a59075" path="/var/lib/kubelet/pods/d7b6148b-0e51-479d-839e-9d8589a59075/volumes" Sep 07 13:09:21.033370 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:21.033331 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:21.096133 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:21.096089 2576 patch_prober.go:28] interesting pod/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 07 13:09:21.096326 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:21.096150 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 07 13:09:22.179354 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:22.179321 2576 generic.go:356] "Generic (PLEG): container finished" podID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerID="aa84c1e5b7dcdae4a453966baf67eccc7402607e4e16ab1afdbff763919b4bd1" exitCode=0 Sep 07 13:09:22.179832 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:22.179379 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" event={"ID":"a325cbe3-b411-4970-889c-91f05a8bb49f","Type":"ContainerDied","Data":"aa84c1e5b7dcdae4a453966baf67eccc7402607e4e16ab1afdbff763919b4bd1"} Sep 07 13:09:22.179832 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:22.179775 2576 scope.go:117] "RemoveContainer" containerID="aa84c1e5b7dcdae4a453966baf67eccc7402607e4e16ab1afdbff763919b4bd1" Sep 07 13:09:22.180813 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:22.180786 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:09:22.180902 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:22.180853 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:09:22.180902 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:22.180883 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:09:23.193656 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.193611 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" event={"ID":"a325cbe3-b411-4970-889c-91f05a8bb49f","Type":"ContainerStarted","Data":"628a52ce9bd637b42778b889a1345e4897f72180e23f9e169b34e2d238be9deb"} Sep 07 13:09:23.818936 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.818914 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:09:23.888996 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.888241 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-istio-envoy\") pod \"931c134a-aecc-410f-ac84-64836b04bd09\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " Sep 07 13:09:23.888996 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.888296 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-istio-data\") pod \"931c134a-aecc-410f-ac84-64836b04bd09\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " Sep 07 13:09:23.888996 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.888325 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-workload-socket\") pod \"931c134a-aecc-410f-ac84-64836b04bd09\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " Sep 07 13:09:23.888996 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.888405 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/931c134a-aecc-410f-ac84-64836b04bd09-istiod-ca-cert\") pod \"931c134a-aecc-410f-ac84-64836b04bd09\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " Sep 07 13:09:23.888996 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.888448 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/931c134a-aecc-410f-ac84-64836b04bd09-istio-podinfo\") pod \"931c134a-aecc-410f-ac84-64836b04bd09\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " Sep 07 13:09:23.888996 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.888472 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/931c134a-aecc-410f-ac84-64836b04bd09-istio-token\") pod \"931c134a-aecc-410f-ac84-64836b04bd09\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " Sep 07 13:09:23.888996 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.888498 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkjth\" (UniqueName: \"kubernetes.io/projected/931c134a-aecc-410f-ac84-64836b04bd09-kube-api-access-dkjth\") pod \"931c134a-aecc-410f-ac84-64836b04bd09\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " Sep 07 13:09:23.888996 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.888554 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-credential-socket\") pod \"931c134a-aecc-410f-ac84-64836b04bd09\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " Sep 07 13:09:23.888996 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.888620 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-workload-certs\") pod \"931c134a-aecc-410f-ac84-64836b04bd09\" (UID: \"931c134a-aecc-410f-ac84-64836b04bd09\") " Sep 07 13:09:23.891995 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.889765 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/931c134a-aecc-410f-ac84-64836b04bd09-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "931c134a-aecc-410f-ac84-64836b04bd09" (UID: "931c134a-aecc-410f-ac84-64836b04bd09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:09:23.891995 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.890076 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "931c134a-aecc-410f-ac84-64836b04bd09" (UID: "931c134a-aecc-410f-ac84-64836b04bd09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:09:23.891995 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.890454 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "931c134a-aecc-410f-ac84-64836b04bd09" (UID: "931c134a-aecc-410f-ac84-64836b04bd09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:09:23.891995 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.890612 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-istio-data" (OuterVolumeSpecName: "istio-data") pod "931c134a-aecc-410f-ac84-64836b04bd09" (UID: "931c134a-aecc-410f-ac84-64836b04bd09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:09:23.891995 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.890748 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "931c134a-aecc-410f-ac84-64836b04bd09" (UID: "931c134a-aecc-410f-ac84-64836b04bd09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:09:23.894558 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.894528 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "931c134a-aecc-410f-ac84-64836b04bd09" (UID: "931c134a-aecc-410f-ac84-64836b04bd09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:09:23.894687 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.894614 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/931c134a-aecc-410f-ac84-64836b04bd09-kube-api-access-dkjth" (OuterVolumeSpecName: "kube-api-access-dkjth") pod "931c134a-aecc-410f-ac84-64836b04bd09" (UID: "931c134a-aecc-410f-ac84-64836b04bd09"). InnerVolumeSpecName "kube-api-access-dkjth". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:09:23.895215 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.895193 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/931c134a-aecc-410f-ac84-64836b04bd09-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "931c134a-aecc-410f-ac84-64836b04bd09" (UID: "931c134a-aecc-410f-ac84-64836b04bd09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 13:09:23.895302 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.895229 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/931c134a-aecc-410f-ac84-64836b04bd09-istio-token" (OuterVolumeSpecName: "istio-token") pod "931c134a-aecc-410f-ac84-64836b04bd09" (UID: "931c134a-aecc-410f-ac84-64836b04bd09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:09:23.990483 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.990327 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-workload-certs\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:09:23.990483 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.990365 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-istio-envoy\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:09:23.990483 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.990379 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-istio-data\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:09:23.990483 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.990395 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-workload-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:09:23.990483 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.990407 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/931c134a-aecc-410f-ac84-64836b04bd09-istiod-ca-cert\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:09:23.990483 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.990420 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/931c134a-aecc-410f-ac84-64836b04bd09-istio-podinfo\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:09:23.990483 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.990431 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/931c134a-aecc-410f-ac84-64836b04bd09-istio-token\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:09:23.990483 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.990444 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkjth\" (UniqueName: \"kubernetes.io/projected/931c134a-aecc-410f-ac84-64836b04bd09-kube-api-access-dkjth\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:09:23.990483 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:23.990457 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/931c134a-aecc-410f-ac84-64836b04bd09-credential-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:09:24.010513 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.010481 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:24.010712 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.010690 2576 patch_prober.go:28] interesting pod/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Sep 07 13:09:24.010793 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.010747 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Sep 07 13:09:24.200881 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.200796 2576 generic.go:356] "Generic (PLEG): container finished" podID="931c134a-aecc-410f-ac84-64836b04bd09" containerID="fb4c3ec4b1e8e23b6aad7cf34e4f265623742a4898d1e7fbe344386d920a7ae3" exitCode=0 Sep 07 13:09:24.200881 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.200839 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" Sep 07 13:09:24.201413 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.200880 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" event={"ID":"931c134a-aecc-410f-ac84-64836b04bd09","Type":"ContainerDied","Data":"fb4c3ec4b1e8e23b6aad7cf34e4f265623742a4898d1e7fbe344386d920a7ae3"} Sep 07 13:09:24.201413 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.200934 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq" event={"ID":"931c134a-aecc-410f-ac84-64836b04bd09","Type":"ContainerDied","Data":"150a6c25a94fed4e55c85ec7445d7ec1ac8d2736ee5960b47953e6e7beaebd9f"} Sep 07 13:09:24.201413 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.200964 2576 scope.go:117] "RemoveContainer" containerID="fb4c3ec4b1e8e23b6aad7cf34e4f265623742a4898d1e7fbe344386d920a7ae3" Sep 07 13:09:24.210759 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.210739 2576 scope.go:117] "RemoveContainer" containerID="6c13709e6b17a35d0fc032ea12db9c1bfcac3977b62f5e4135451fb623971b99" Sep 07 13:09:24.218890 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.218874 2576 scope.go:117] "RemoveContainer" containerID="fb4c3ec4b1e8e23b6aad7cf34e4f265623742a4898d1e7fbe344386d920a7ae3" Sep 07 13:09:24.219175 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:09:24.219156 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb4c3ec4b1e8e23b6aad7cf34e4f265623742a4898d1e7fbe344386d920a7ae3\": container with ID starting with fb4c3ec4b1e8e23b6aad7cf34e4f265623742a4898d1e7fbe344386d920a7ae3 not found: ID does not exist" containerID="fb4c3ec4b1e8e23b6aad7cf34e4f265623742a4898d1e7fbe344386d920a7ae3" Sep 07 13:09:24.219224 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.219183 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb4c3ec4b1e8e23b6aad7cf34e4f265623742a4898d1e7fbe344386d920a7ae3"} err="failed to get container status \"fb4c3ec4b1e8e23b6aad7cf34e4f265623742a4898d1e7fbe344386d920a7ae3\": rpc error: code = NotFound desc = could not find container \"fb4c3ec4b1e8e23b6aad7cf34e4f265623742a4898d1e7fbe344386d920a7ae3\": container with ID starting with fb4c3ec4b1e8e23b6aad7cf34e4f265623742a4898d1e7fbe344386d920a7ae3 not found: ID does not exist" Sep 07 13:09:24.219224 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.219200 2576 scope.go:117] "RemoveContainer" containerID="6c13709e6b17a35d0fc032ea12db9c1bfcac3977b62f5e4135451fb623971b99" Sep 07 13:09:24.219454 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:09:24.219437 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c13709e6b17a35d0fc032ea12db9c1bfcac3977b62f5e4135451fb623971b99\": container with ID starting with 6c13709e6b17a35d0fc032ea12db9c1bfcac3977b62f5e4135451fb623971b99 not found: ID does not exist" containerID="6c13709e6b17a35d0fc032ea12db9c1bfcac3977b62f5e4135451fb623971b99" Sep 07 13:09:24.219506 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.219460 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c13709e6b17a35d0fc032ea12db9c1bfcac3977b62f5e4135451fb623971b99"} err="failed to get container status \"6c13709e6b17a35d0fc032ea12db9c1bfcac3977b62f5e4135451fb623971b99\": rpc error: code = NotFound desc = could not find container \"6c13709e6b17a35d0fc032ea12db9c1bfcac3977b62f5e4135451fb623971b99\": container with ID starting with 6c13709e6b17a35d0fc032ea12db9c1bfcac3977b62f5e4135451fb623971b99 not found: ID does not exist" Sep 07 13:09:24.222206 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.222184 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq"] Sep 07 13:09:24.229056 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.229038 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-42173418-openshift-default-8645f8b54d-gt6pq"] Sep 07 13:09:24.488045 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:24.487926 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="931c134a-aecc-410f-ac84-64836b04bd09" path="/var/lib/kubelet/pods/931c134a-aecc-410f-ac84-64836b04bd09/volumes" Sep 07 13:09:25.010928 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:25.010887 2576 patch_prober.go:28] interesting pod/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Sep 07 13:09:25.011139 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:25.010987 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Sep 07 13:09:26.010534 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:26.010498 2576 patch_prober.go:28] interesting pod/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Sep 07 13:09:26.010930 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:26.010564 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Sep 07 13:09:27.011369 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:27.011334 2576 patch_prober.go:28] interesting pod/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Sep 07 13:09:27.011750 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:27.011388 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Sep 07 13:09:27.848492 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:27.848451 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:27.923983 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:27.923931 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:28.010387 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:28.010350 2576 patch_prober.go:28] interesting pod/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Sep 07 13:09:28.010576 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:28.010417 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Sep 07 13:09:28.023244 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:28.023216 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:28.123665 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:28.123572 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:28.223469 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:28.223432 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:28.326926 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:28.326891 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:28.604290 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:28.604258 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:29.050083 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:29.050054 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:29.050540 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:29.050393 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:29.051371 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:29.051354 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:09:29.247172 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:29.247136 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:31.107254 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:31.107216 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:50.530408 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:50.530371 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:53.405829 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.405786 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:53.508471 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.508438 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q"] Sep 07 13:09:53.508816 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.508804 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="931c134a-aecc-410f-ac84-64836b04bd09" containerName="istio-proxy" Sep 07 13:09:53.508873 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.508817 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="931c134a-aecc-410f-ac84-64836b04bd09" containerName="istio-proxy" Sep 07 13:09:53.508873 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.508833 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7b6148b-0e51-479d-839e-9d8589a59075" containerName="payload-pre-processing" Sep 07 13:09:53.508873 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.508838 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7b6148b-0e51-479d-839e-9d8589a59075" containerName="payload-pre-processing" Sep 07 13:09:53.508990 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.508898 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7b6148b-0e51-479d-839e-9d8589a59075" containerName="payload-pre-processing" Sep 07 13:09:53.508990 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.508908 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="931c134a-aecc-410f-ac84-64836b04bd09" containerName="istio-proxy" Sep 07 13:09:53.508990 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.508915 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="931c134a-aecc-410f-ac84-64836b04bd09" containerName="istio-proxy" Sep 07 13:09:53.513091 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.513074 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q" Sep 07 13:09:53.514920 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.514898 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 13:09:53.515402 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.515177 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-flngs\"" Sep 07 13:09:53.520154 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.520135 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q"] Sep 07 13:09:53.559645 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.559606 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/881d00e3-b2be-468d-a086-e34e78ee5b91-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q\" (UID: \"881d00e3-b2be-468d-a086-e34e78ee5b91\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q" Sep 07 13:09:53.661054 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.660935 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/881d00e3-b2be-468d-a086-e34e78ee5b91-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q\" (UID: \"881d00e3-b2be-468d-a086-e34e78ee5b91\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q" Sep 07 13:09:53.661596 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.661573 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/881d00e3-b2be-468d-a086-e34e78ee5b91-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q\" (UID: \"881d00e3-b2be-468d-a086-e34e78ee5b91\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q" Sep 07 13:09:53.826742 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:53.826708 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q" Sep 07 13:09:54.102207 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:54.102180 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q"] Sep 07 13:09:54.104483 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:09:54.104458 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881d00e3_b2be_468d_a086_e34e78ee5b91.slice/crio-d16739dae0a68e8174778143ad0f54fbac0fa34c91cddee64b110d887bb5560f WatchSource:0}: Error finding container d16739dae0a68e8174778143ad0f54fbac0fa34c91cddee64b110d887bb5560f: Status 404 returned error can't find the container with id d16739dae0a68e8174778143ad0f54fbac0fa34c91cddee64b110d887bb5560f Sep 07 13:09:54.347823 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:54.347787 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q" event={"ID":"881d00e3-b2be-468d-a086-e34e78ee5b91","Type":"ContainerStarted","Data":"d16739dae0a68e8174778143ad0f54fbac0fa34c91cddee64b110d887bb5560f"} Sep 07 13:09:54.532399 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:54.532358 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:09:55.353823 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:55.353782 2576 generic.go:356] "Generic (PLEG): container finished" podID="881d00e3-b2be-468d-a086-e34e78ee5b91" containerID="d771da394f1aae0384521434906fec5c451ba215a3f1ab276eeb4fef556d6f87" exitCode=0 Sep 07 13:09:55.354012 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:55.353875 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q" event={"ID":"881d00e3-b2be-468d-a086-e34e78ee5b91","Type":"ContainerDied","Data":"d771da394f1aae0384521434906fec5c451ba215a3f1ab276eeb4fef556d6f87"} Sep 07 13:09:56.498347 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:56.498321 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q" Sep 07 13:09:56.588933 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:56.588898 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/881d00e3-b2be-468d-a086-e34e78ee5b91-maas-api-service-ca\") pod \"881d00e3-b2be-468d-a086-e34e78ee5b91\" (UID: \"881d00e3-b2be-468d-a086-e34e78ee5b91\") " Sep 07 13:09:56.589299 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:56.589274 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/881d00e3-b2be-468d-a086-e34e78ee5b91-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "881d00e3-b2be-468d-a086-e34e78ee5b91" (UID: "881d00e3-b2be-468d-a086-e34e78ee5b91"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:09:56.689703 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:56.689599 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/881d00e3-b2be-468d-a086-e34e78ee5b91-maas-api-service-ca\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:09:57.365776 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:57.365737 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q" event={"ID":"881d00e3-b2be-468d-a086-e34e78ee5b91","Type":"ContainerDied","Data":"d16739dae0a68e8174778143ad0f54fbac0fa34c91cddee64b110d887bb5560f"} Sep 07 13:09:57.365776 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:57.365763 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q" Sep 07 13:09:57.366014 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:09:57.365793 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d16739dae0a68e8174778143ad0f54fbac0fa34c91cddee64b110d887bb5560f" Sep 07 13:10:01.358103 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:01.358061 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:01.466944 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:01.466911 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:01.574148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:01.574115 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:04.223556 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:04.223519 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:04.455703 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:04.455667 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q"] Sep 07 13:10:04.460905 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:04.460878 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-02ea24-8gq2q"] Sep 07 13:10:04.482543 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:04.482478 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="881d00e3-b2be-468d-a086-e34e78ee5b91" path="/var/lib/kubelet/pods/881d00e3-b2be-468d-a086-e34e78ee5b91/volumes" Sep 07 13:10:06.407248 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:06.407197 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:06.514059 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:06.514016 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:09.904406 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:09.904355 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:11.625923 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:11.625893 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f"] Sep 07 13:10:11.626389 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:11.626155 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" containerID="cri-o://628a52ce9bd637b42778b889a1345e4897f72180e23f9e169b34e2d238be9deb" gracePeriod=30 Sep 07 13:10:13.127134 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:13.127097 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:13.226377 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:13.226338 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:13.977987 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:13.977918 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:14.052717 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:14.052682 2576 patch_prober.go:28] interesting pod/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 07 13:10:14.052886 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:14.052757 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 13:10:14.076855 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:14.076822 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:14.195751 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:14.195721 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:14.910568 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:14.910537 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:14.981526 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:14.981490 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:16.884164 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.884125 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:10:16.980544 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.980456 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-workload-socket\") pod \"a325cbe3-b411-4970-889c-91f05a8bb49f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " Sep 07 13:10:16.980544 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.980495 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-credential-socket\") pod \"a325cbe3-b411-4970-889c-91f05a8bb49f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " Sep 07 13:10:16.980544 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.980536 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-token\") pod \"a325cbe3-b411-4970-889c-91f05a8bb49f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " Sep 07 13:10:16.980801 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.980560 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-workload-certs\") pod \"a325cbe3-b411-4970-889c-91f05a8bb49f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " Sep 07 13:10:16.980801 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.980593 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw7xg\" (UniqueName: \"kubernetes.io/projected/a325cbe3-b411-4970-889c-91f05a8bb49f-kube-api-access-hw7xg\") pod \"a325cbe3-b411-4970-889c-91f05a8bb49f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " Sep 07 13:10:16.980801 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.980617 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-podinfo\") pod \"a325cbe3-b411-4970-889c-91f05a8bb49f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " Sep 07 13:10:16.980801 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.980668 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-envoy\") pod \"a325cbe3-b411-4970-889c-91f05a8bb49f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " Sep 07 13:10:16.980801 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.980697 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-data\") pod \"a325cbe3-b411-4970-889c-91f05a8bb49f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " Sep 07 13:10:16.980801 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.980729 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a325cbe3-b411-4970-889c-91f05a8bb49f-istiod-ca-cert\") pod \"a325cbe3-b411-4970-889c-91f05a8bb49f\" (UID: \"a325cbe3-b411-4970-889c-91f05a8bb49f\") " Sep 07 13:10:16.981157 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.980795 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a325cbe3-b411-4970-889c-91f05a8bb49f" (UID: "a325cbe3-b411-4970-889c-91f05a8bb49f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:10:16.981157 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.980813 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a325cbe3-b411-4970-889c-91f05a8bb49f" (UID: "a325cbe3-b411-4970-889c-91f05a8bb49f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:10:16.981157 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.980939 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a325cbe3-b411-4970-889c-91f05a8bb49f" (UID: "a325cbe3-b411-4970-889c-91f05a8bb49f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:10:16.981157 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.981043 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-data" (OuterVolumeSpecName: "istio-data") pod "a325cbe3-b411-4970-889c-91f05a8bb49f" (UID: "a325cbe3-b411-4970-889c-91f05a8bb49f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:10:16.981157 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.981105 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-data\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:16.981157 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.981117 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-workload-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:16.981157 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.981129 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-credential-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:16.981157 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.981141 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-workload-certs\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:16.981719 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.981161 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a325cbe3-b411-4970-889c-91f05a8bb49f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a325cbe3-b411-4970-889c-91f05a8bb49f" (UID: "a325cbe3-b411-4970-889c-91f05a8bb49f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:10:16.983183 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.983152 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-token" (OuterVolumeSpecName: "istio-token") pod "a325cbe3-b411-4970-889c-91f05a8bb49f" (UID: "a325cbe3-b411-4970-889c-91f05a8bb49f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:10:16.983183 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.983170 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a325cbe3-b411-4970-889c-91f05a8bb49f" (UID: "a325cbe3-b411-4970-889c-91f05a8bb49f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:10:16.983560 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.983534 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a325cbe3-b411-4970-889c-91f05a8bb49f" (UID: "a325cbe3-b411-4970-889c-91f05a8bb49f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 13:10:16.983560 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:16.983549 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a325cbe3-b411-4970-889c-91f05a8bb49f-kube-api-access-hw7xg" (OuterVolumeSpecName: "kube-api-access-hw7xg") pod "a325cbe3-b411-4970-889c-91f05a8bb49f" (UID: "a325cbe3-b411-4970-889c-91f05a8bb49f"). InnerVolumeSpecName "kube-api-access-hw7xg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:10:17.081868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.081830 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a325cbe3-b411-4970-889c-91f05a8bb49f-istiod-ca-cert\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:17.081868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.081860 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-token\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:17.081868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.081869 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw7xg\" (UniqueName: \"kubernetes.io/projected/a325cbe3-b411-4970-889c-91f05a8bb49f-kube-api-access-hw7xg\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:17.081868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.081878 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-podinfo\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:17.082165 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.081887 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a325cbe3-b411-4970-889c-91f05a8bb49f-istio-envoy\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:17.475222 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.475178 2576 generic.go:356] "Generic (PLEG): container finished" podID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerID="628a52ce9bd637b42778b889a1345e4897f72180e23f9e169b34e2d238be9deb" exitCode=0 Sep 07 13:10:17.475430 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.475249 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" Sep 07 13:10:17.475430 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.475262 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" event={"ID":"a325cbe3-b411-4970-889c-91f05a8bb49f","Type":"ContainerDied","Data":"628a52ce9bd637b42778b889a1345e4897f72180e23f9e169b34e2d238be9deb"} Sep 07 13:10:17.475430 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.475339 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f" event={"ID":"a325cbe3-b411-4970-889c-91f05a8bb49f","Type":"ContainerDied","Data":"7b713788c685db359c32fc04133d1d1923b271b1ef641523541136babceeebca"} Sep 07 13:10:17.475430 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.475362 2576 scope.go:117] "RemoveContainer" containerID="628a52ce9bd637b42778b889a1345e4897f72180e23f9e169b34e2d238be9deb" Sep 07 13:10:17.485423 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.485404 2576 scope.go:117] "RemoveContainer" containerID="aa84c1e5b7dcdae4a453966baf67eccc7402607e4e16ab1afdbff763919b4bd1" Sep 07 13:10:17.493549 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.493532 2576 scope.go:117] "RemoveContainer" containerID="628a52ce9bd637b42778b889a1345e4897f72180e23f9e169b34e2d238be9deb" Sep 07 13:10:17.493820 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:10:17.493804 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"628a52ce9bd637b42778b889a1345e4897f72180e23f9e169b34e2d238be9deb\": container with ID starting with 628a52ce9bd637b42778b889a1345e4897f72180e23f9e169b34e2d238be9deb not found: ID does not exist" containerID="628a52ce9bd637b42778b889a1345e4897f72180e23f9e169b34e2d238be9deb" Sep 07 13:10:17.493865 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.493828 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"628a52ce9bd637b42778b889a1345e4897f72180e23f9e169b34e2d238be9deb"} err="failed to get container status \"628a52ce9bd637b42778b889a1345e4897f72180e23f9e169b34e2d238be9deb\": rpc error: code = NotFound desc = could not find container \"628a52ce9bd637b42778b889a1345e4897f72180e23f9e169b34e2d238be9deb\": container with ID starting with 628a52ce9bd637b42778b889a1345e4897f72180e23f9e169b34e2d238be9deb not found: ID does not exist" Sep 07 13:10:17.493865 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.493845 2576 scope.go:117] "RemoveContainer" containerID="aa84c1e5b7dcdae4a453966baf67eccc7402607e4e16ab1afdbff763919b4bd1" Sep 07 13:10:17.494109 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:10:17.494081 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa84c1e5b7dcdae4a453966baf67eccc7402607e4e16ab1afdbff763919b4bd1\": container with ID starting with aa84c1e5b7dcdae4a453966baf67eccc7402607e4e16ab1afdbff763919b4bd1 not found: ID does not exist" containerID="aa84c1e5b7dcdae4a453966baf67eccc7402607e4e16ab1afdbff763919b4bd1" Sep 07 13:10:17.494153 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.494112 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa84c1e5b7dcdae4a453966baf67eccc7402607e4e16ab1afdbff763919b4bd1"} err="failed to get container status \"aa84c1e5b7dcdae4a453966baf67eccc7402607e4e16ab1afdbff763919b4bd1\": rpc error: code = NotFound desc = could not find container \"aa84c1e5b7dcdae4a453966baf67eccc7402607e4e16ab1afdbff763919b4bd1\": container with ID starting with aa84c1e5b7dcdae4a453966baf67eccc7402607e4e16ab1afdbff763919b4bd1 not found: ID does not exist" Sep 07 13:10:17.504093 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.504065 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:17.512303 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.512273 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f"] Sep 07 13:10:17.517176 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:17.517154 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-02ea24-openshift-default-6b47bfbdb4-97j2f"] Sep 07 13:10:18.483155 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:18.483114 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" path="/var/lib/kubelet/pods/a325cbe3-b411-4970-889c-91f05a8bb49f/volumes" Sep 07 13:10:20.729101 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:20.729062 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:23.943748 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:23.943711 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:24.010190 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:24.010153 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:24.099488 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:24.099448 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:24.196050 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:24.195942 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:25.535690 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:25.535632 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:31.023053 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:31.023012 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:31.142422 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:31.142384 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:31.385449 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:31.385419 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:31.490318 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:31.490281 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:34.703898 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:34.703863 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:34.802375 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:34.802330 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:35.539093 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:35.539039 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:35.999338 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:35.999250 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:36.013909 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.013877 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84"] Sep 07 13:10:36.014326 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.014311 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" Sep 07 13:10:36.014379 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.014329 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" Sep 07 13:10:36.014379 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.014341 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="931c134a-aecc-410f-ac84-64836b04bd09" containerName="istio-proxy" Sep 07 13:10:36.014379 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.014346 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="931c134a-aecc-410f-ac84-64836b04bd09" containerName="istio-proxy" Sep 07 13:10:36.014379 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.014359 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="881d00e3-b2be-468d-a086-e34e78ee5b91" containerName="revoke-keys" Sep 07 13:10:36.014379 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.014364 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="881d00e3-b2be-468d-a086-e34e78ee5b91" containerName="revoke-keys" Sep 07 13:10:36.014535 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.014420 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="881d00e3-b2be-468d-a086-e34e78ee5b91" containerName="revoke-keys" Sep 07 13:10:36.014535 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.014431 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" Sep 07 13:10:36.014535 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.014438 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" Sep 07 13:10:36.019183 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.019168 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84" Sep 07 13:10:36.021308 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.021288 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-flngs\"" Sep 07 13:10:36.021486 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.021464 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 13:10:36.031681 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.031650 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84"] Sep 07 13:10:36.046833 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.046806 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4ac1122-03b2-4a55-8c51-96fa1fa375ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84\" (UID: \"b4ac1122-03b2-4a55-8c51-96fa1fa375ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84" Sep 07 13:10:36.148128 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.148089 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4ac1122-03b2-4a55-8c51-96fa1fa375ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84\" (UID: \"b4ac1122-03b2-4a55-8c51-96fa1fa375ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84" Sep 07 13:10:36.148744 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.148722 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4ac1122-03b2-4a55-8c51-96fa1fa375ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84\" (UID: \"b4ac1122-03b2-4a55-8c51-96fa1fa375ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84" Sep 07 13:10:36.223759 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.223716 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:36.329758 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.329730 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84" Sep 07 13:10:36.566302 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.566269 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:36.605724 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:36.605698 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84"] Sep 07 13:10:36.608052 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:10:36.608026 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ac1122_03b2_4a55_8c51_96fa1fa375ad.slice/crio-2e3f8ed02233ac33b20d1473f1441485919059731857a2840bfbe5ba7e63b972 WatchSource:0}: Error finding container 2e3f8ed02233ac33b20d1473f1441485919059731857a2840bfbe5ba7e63b972: Status 404 returned error can't find the container with id 2e3f8ed02233ac33b20d1473f1441485919059731857a2840bfbe5ba7e63b972 Sep 07 13:10:37.573263 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:37.573184 2576 generic.go:356] "Generic (PLEG): container finished" podID="b4ac1122-03b2-4a55-8c51-96fa1fa375ad" containerID="a56f81841c99322b182071bd66f10920cb22ec133dac003fbe28852aa538a509" exitCode=0 Sep 07 13:10:37.573644 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:37.573268 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84" event={"ID":"b4ac1122-03b2-4a55-8c51-96fa1fa375ad","Type":"ContainerDied","Data":"a56f81841c99322b182071bd66f10920cb22ec133dac003fbe28852aa538a509"} Sep 07 13:10:37.573644 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:37.573342 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84" event={"ID":"b4ac1122-03b2-4a55-8c51-96fa1fa375ad","Type":"ContainerStarted","Data":"2e3f8ed02233ac33b20d1473f1441485919059731857a2840bfbe5ba7e63b972"} Sep 07 13:10:38.708074 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:38.708048 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84" Sep 07 13:10:38.769053 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:38.769022 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4ac1122-03b2-4a55-8c51-96fa1fa375ad-maas-api-service-ca\") pod \"b4ac1122-03b2-4a55-8c51-96fa1fa375ad\" (UID: \"b4ac1122-03b2-4a55-8c51-96fa1fa375ad\") " Sep 07 13:10:38.769351 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:38.769327 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4ac1122-03b2-4a55-8c51-96fa1fa375ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b4ac1122-03b2-4a55-8c51-96fa1fa375ad" (UID: "b4ac1122-03b2-4a55-8c51-96fa1fa375ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:10:38.870017 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:38.869912 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4ac1122-03b2-4a55-8c51-96fa1fa375ad-maas-api-service-ca\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:39.584116 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:39.584086 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84" Sep 07 13:10:39.584300 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:39.584084 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84" event={"ID":"b4ac1122-03b2-4a55-8c51-96fa1fa375ad","Type":"ContainerDied","Data":"2e3f8ed02233ac33b20d1473f1441485919059731857a2840bfbe5ba7e63b972"} Sep 07 13:10:39.584300 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:39.584209 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e3f8ed02233ac33b20d1473f1441485919059731857a2840bfbe5ba7e63b972" Sep 07 13:10:42.752033 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:42.752000 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:42.856896 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:42.856858 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:42.951903 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:42.951858 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:43.053440 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:43.053405 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:43.657423 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:43.657384 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:45.459091 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:45.459058 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:45.524852 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:45.524814 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:46.721186 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:46.721131 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:46.755879 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:46.755846 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm"] Sep 07 13:10:46.756157 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:46.756127 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" podUID="f7866a05-0fee-4b7e-93de-3e6512df788c" containerName="payload-processing" containerID="cri-o://6ab4e874f10bec38a70310852d2a68cca6799d42cfdbb1d587fcf3e1b5eb7ed1" gracePeriod=30 Sep 07 13:10:46.763556 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:46.763530 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv"] Sep 07 13:10:46.763921 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:46.763875 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" podUID="64f34894-7c4f-4bcb-832d-d92aeef8710f" containerName="payload-pre-processing" containerID="cri-o://b32fa950b36a7cdb4802ab3126b6d61fb4603b741efe5773d340fc5965738952" gracePeriod=30 Sep 07 13:10:46.794303 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:46.794274 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:46.895166 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:46.895128 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:46.930717 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:46.930683 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84"] Sep 07 13:10:46.934119 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:46.934096 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7ce94d-pnd84"] Sep 07 13:10:46.959075 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:46.959041 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:47.046271 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.046247 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" Sep 07 13:10:47.048817 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.048801 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" Sep 07 13:10:47.144422 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.144380 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4gjw\" (UniqueName: \"kubernetes.io/projected/64f34894-7c4f-4bcb-832d-d92aeef8710f-kube-api-access-q4gjw\") pod \"64f34894-7c4f-4bcb-832d-d92aeef8710f\" (UID: \"64f34894-7c4f-4bcb-832d-d92aeef8710f\") " Sep 07 13:10:47.144422 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.144426 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjv76\" (UniqueName: \"kubernetes.io/projected/f7866a05-0fee-4b7e-93de-3e6512df788c-kube-api-access-bjv76\") pod \"f7866a05-0fee-4b7e-93de-3e6512df788c\" (UID: \"f7866a05-0fee-4b7e-93de-3e6512df788c\") " Sep 07 13:10:47.144680 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.144471 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7866a05-0fee-4b7e-93de-3e6512df788c-plugins-config-volume\") pod \"f7866a05-0fee-4b7e-93de-3e6512df788c\" (UID: \"f7866a05-0fee-4b7e-93de-3e6512df788c\") " Sep 07 13:10:47.144680 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.144552 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64f34894-7c4f-4bcb-832d-d92aeef8710f-plugins-config-volume\") pod \"64f34894-7c4f-4bcb-832d-d92aeef8710f\" (UID: \"64f34894-7c4f-4bcb-832d-d92aeef8710f\") " Sep 07 13:10:47.144912 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.144883 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7866a05-0fee-4b7e-93de-3e6512df788c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7866a05-0fee-4b7e-93de-3e6512df788c" (UID: "f7866a05-0fee-4b7e-93de-3e6512df788c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:10:47.145010 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.144944 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64f34894-7c4f-4bcb-832d-d92aeef8710f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "64f34894-7c4f-4bcb-832d-d92aeef8710f" (UID: "64f34894-7c4f-4bcb-832d-d92aeef8710f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:10:47.146788 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.146766 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64f34894-7c4f-4bcb-832d-d92aeef8710f-kube-api-access-q4gjw" (OuterVolumeSpecName: "kube-api-access-q4gjw") pod "64f34894-7c4f-4bcb-832d-d92aeef8710f" (UID: "64f34894-7c4f-4bcb-832d-d92aeef8710f"). InnerVolumeSpecName "kube-api-access-q4gjw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:10:47.146886 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.146788 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7866a05-0fee-4b7e-93de-3e6512df788c-kube-api-access-bjv76" (OuterVolumeSpecName: "kube-api-access-bjv76") pod "f7866a05-0fee-4b7e-93de-3e6512df788c" (UID: "f7866a05-0fee-4b7e-93de-3e6512df788c"). InnerVolumeSpecName "kube-api-access-bjv76". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:10:47.246310 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.246265 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64f34894-7c4f-4bcb-832d-d92aeef8710f-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:47.246310 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.246304 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4gjw\" (UniqueName: \"kubernetes.io/projected/64f34894-7c4f-4bcb-832d-d92aeef8710f-kube-api-access-q4gjw\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:47.246310 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.246314 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjv76\" (UniqueName: \"kubernetes.io/projected/f7866a05-0fee-4b7e-93de-3e6512df788c-kube-api-access-bjv76\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:47.246310 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.246325 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7866a05-0fee-4b7e-93de-3e6512df788c-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:47.388188 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.388100 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:47.626989 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.626933 2576 generic.go:356] "Generic (PLEG): container finished" podID="f7866a05-0fee-4b7e-93de-3e6512df788c" containerID="6ab4e874f10bec38a70310852d2a68cca6799d42cfdbb1d587fcf3e1b5eb7ed1" exitCode=0 Sep 07 13:10:47.627183 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.627000 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" Sep 07 13:10:47.627183 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.627024 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" event={"ID":"f7866a05-0fee-4b7e-93de-3e6512df788c","Type":"ContainerDied","Data":"6ab4e874f10bec38a70310852d2a68cca6799d42cfdbb1d587fcf3e1b5eb7ed1"} Sep 07 13:10:47.627183 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.627074 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm" event={"ID":"f7866a05-0fee-4b7e-93de-3e6512df788c","Type":"ContainerDied","Data":"78bdf28c065a430fe5f68a8f76d10d2f53fe44c6c51468f695e33681398e146e"} Sep 07 13:10:47.627183 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.627098 2576 scope.go:117] "RemoveContainer" containerID="6ab4e874f10bec38a70310852d2a68cca6799d42cfdbb1d587fcf3e1b5eb7ed1" Sep 07 13:10:47.628491 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.628365 2576 generic.go:356] "Generic (PLEG): container finished" podID="64f34894-7c4f-4bcb-832d-d92aeef8710f" containerID="b32fa950b36a7cdb4802ab3126b6d61fb4603b741efe5773d340fc5965738952" exitCode=0 Sep 07 13:10:47.628491 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.628417 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" Sep 07 13:10:47.628491 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.628427 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" event={"ID":"64f34894-7c4f-4bcb-832d-d92aeef8710f","Type":"ContainerDied","Data":"b32fa950b36a7cdb4802ab3126b6d61fb4603b741efe5773d340fc5965738952"} Sep 07 13:10:47.628491 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.628465 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv" event={"ID":"64f34894-7c4f-4bcb-832d-d92aeef8710f","Type":"ContainerDied","Data":"0289f6c52c932bbe08ce89ecd18775b891b5e94a70baf47196a729fb6ec4eb66"} Sep 07 13:10:47.636304 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.636283 2576 scope.go:117] "RemoveContainer" containerID="6ab4e874f10bec38a70310852d2a68cca6799d42cfdbb1d587fcf3e1b5eb7ed1" Sep 07 13:10:47.636598 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:10:47.636579 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ab4e874f10bec38a70310852d2a68cca6799d42cfdbb1d587fcf3e1b5eb7ed1\": container with ID starting with 6ab4e874f10bec38a70310852d2a68cca6799d42cfdbb1d587fcf3e1b5eb7ed1 not found: ID does not exist" containerID="6ab4e874f10bec38a70310852d2a68cca6799d42cfdbb1d587fcf3e1b5eb7ed1" Sep 07 13:10:47.636659 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.636608 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ab4e874f10bec38a70310852d2a68cca6799d42cfdbb1d587fcf3e1b5eb7ed1"} err="failed to get container status \"6ab4e874f10bec38a70310852d2a68cca6799d42cfdbb1d587fcf3e1b5eb7ed1\": rpc error: code = NotFound desc = could not find container \"6ab4e874f10bec38a70310852d2a68cca6799d42cfdbb1d587fcf3e1b5eb7ed1\": container with ID starting with 6ab4e874f10bec38a70310852d2a68cca6799d42cfdbb1d587fcf3e1b5eb7ed1 not found: ID does not exist" Sep 07 13:10:47.636659 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.636624 2576 scope.go:117] "RemoveContainer" containerID="b32fa950b36a7cdb4802ab3126b6d61fb4603b741efe5773d340fc5965738952" Sep 07 13:10:47.644756 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.644739 2576 scope.go:117] "RemoveContainer" containerID="b32fa950b36a7cdb4802ab3126b6d61fb4603b741efe5773d340fc5965738952" Sep 07 13:10:47.645013 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:10:47.644995 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b32fa950b36a7cdb4802ab3126b6d61fb4603b741efe5773d340fc5965738952\": container with ID starting with b32fa950b36a7cdb4802ab3126b6d61fb4603b741efe5773d340fc5965738952 not found: ID does not exist" containerID="b32fa950b36a7cdb4802ab3126b6d61fb4603b741efe5773d340fc5965738952" Sep 07 13:10:47.645067 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.645020 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32fa950b36a7cdb4802ab3126b6d61fb4603b741efe5773d340fc5965738952"} err="failed to get container status \"b32fa950b36a7cdb4802ab3126b6d61fb4603b741efe5773d340fc5965738952\": rpc error: code = NotFound desc = could not find container \"b32fa950b36a7cdb4802ab3126b6d61fb4603b741efe5773d340fc5965738952\": container with ID starting with b32fa950b36a7cdb4802ab3126b6d61fb4603b741efe5773d340fc5965738952 not found: ID does not exist" Sep 07 13:10:47.655121 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.655101 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv"] Sep 07 13:10:47.661392 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.661365 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7ce94d-9d8f77b74-jdscv"] Sep 07 13:10:47.675717 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.675692 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm"] Sep 07 13:10:47.684443 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:47.684425 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7ce94d-557cd8cf89-lqwkm"] Sep 07 13:10:48.481901 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:48.481866 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64f34894-7c4f-4bcb-832d-d92aeef8710f" path="/var/lib/kubelet/pods/64f34894-7c4f-4bcb-832d-d92aeef8710f/volumes" Sep 07 13:10:48.482280 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:48.482218 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4ac1122-03b2-4a55-8c51-96fa1fa375ad" path="/var/lib/kubelet/pods/b4ac1122-03b2-4a55-8c51-96fa1fa375ad/volumes" Sep 07 13:10:48.482520 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:48.482507 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7866a05-0fee-4b7e-93de-3e6512df788c" path="/var/lib/kubelet/pods/f7866a05-0fee-4b7e-93de-3e6512df788c/volumes" Sep 07 13:10:52.276115 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:52.276072 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:52.375792 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:52.375752 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:52.960328 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:52.960296 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:54.336915 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.336884 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:54.434284 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.434242 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:54.644577 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.644488 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:54.953716 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.953631 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948"] Sep 07 13:10:54.954100 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.954081 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" Sep 07 13:10:54.954218 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.954102 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a325cbe3-b411-4970-889c-91f05a8bb49f" containerName="istio-proxy" Sep 07 13:10:54.954218 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.954114 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64f34894-7c4f-4bcb-832d-d92aeef8710f" containerName="payload-pre-processing" Sep 07 13:10:54.954218 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.954121 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="64f34894-7c4f-4bcb-832d-d92aeef8710f" containerName="payload-pre-processing" Sep 07 13:10:54.954218 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.954137 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7866a05-0fee-4b7e-93de-3e6512df788c" containerName="payload-processing" Sep 07 13:10:54.954218 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.954145 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7866a05-0fee-4b7e-93de-3e6512df788c" containerName="payload-processing" Sep 07 13:10:54.954218 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.954167 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4ac1122-03b2-4a55-8c51-96fa1fa375ad" containerName="revoke-keys" Sep 07 13:10:54.954218 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.954175 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4ac1122-03b2-4a55-8c51-96fa1fa375ad" containerName="revoke-keys" Sep 07 13:10:54.954552 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.954286 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7866a05-0fee-4b7e-93de-3e6512df788c" containerName="payload-processing" Sep 07 13:10:54.954552 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.954299 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4ac1122-03b2-4a55-8c51-96fa1fa375ad" containerName="revoke-keys" Sep 07 13:10:54.954552 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.954309 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="64f34894-7c4f-4bcb-832d-d92aeef8710f" containerName="payload-pre-processing" Sep 07 13:10:54.958794 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.958775 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948" Sep 07 13:10:54.960786 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.960765 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 13:10:54.960887 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.960831 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-flngs\"" Sep 07 13:10:54.965098 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:54.965077 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948"] Sep 07 13:10:55.068442 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:55.068403 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:10:55.118776 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:55.118730 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07c3c902-175e-405a-9711-63e7dcfc66eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-76f028-hx948\" (UID: \"07c3c902-175e-405a-9711-63e7dcfc66eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948" Sep 07 13:10:55.220020 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:55.219909 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07c3c902-175e-405a-9711-63e7dcfc66eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-76f028-hx948\" (UID: \"07c3c902-175e-405a-9711-63e7dcfc66eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948" Sep 07 13:10:55.220519 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:55.220497 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07c3c902-175e-405a-9711-63e7dcfc66eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-76f028-hx948\" (UID: \"07c3c902-175e-405a-9711-63e7dcfc66eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948" Sep 07 13:10:55.272105 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:55.272067 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948" Sep 07 13:10:55.546417 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:55.546386 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948"] Sep 07 13:10:55.673535 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:55.673478 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948" event={"ID":"07c3c902-175e-405a-9711-63e7dcfc66eb","Type":"ContainerStarted","Data":"418e44d4d326a3488cb57da87b3af42704b390dacb648752a503f931af274d2f"} Sep 07 13:10:56.679648 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:56.679609 2576 generic.go:356] "Generic (PLEG): container finished" podID="07c3c902-175e-405a-9711-63e7dcfc66eb" containerID="2f8df9a0cfd0ab8bcb4b13409465e60d5dc764aa7d1425f9e3484b1a7e8bcff2" exitCode=0 Sep 07 13:10:56.680071 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:56.679662 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948" event={"ID":"07c3c902-175e-405a-9711-63e7dcfc66eb","Type":"ContainerDied","Data":"2f8df9a0cfd0ab8bcb4b13409465e60d5dc764aa7d1425f9e3484b1a7e8bcff2"} Sep 07 13:10:57.824038 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:57.824009 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948" Sep 07 13:10:57.945822 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:57.945726 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07c3c902-175e-405a-9711-63e7dcfc66eb-maas-api-service-ca\") pod \"07c3c902-175e-405a-9711-63e7dcfc66eb\" (UID: \"07c3c902-175e-405a-9711-63e7dcfc66eb\") " Sep 07 13:10:57.946174 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:57.946151 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07c3c902-175e-405a-9711-63e7dcfc66eb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "07c3c902-175e-405a-9711-63e7dcfc66eb" (UID: "07c3c902-175e-405a-9711-63e7dcfc66eb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:10:58.047435 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:58.047400 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07c3c902-175e-405a-9711-63e7dcfc66eb-maas-api-service-ca\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:10:58.690828 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:58.690797 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948" Sep 07 13:10:58.691043 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:58.690796 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948" event={"ID":"07c3c902-175e-405a-9711-63e7dcfc66eb","Type":"ContainerDied","Data":"418e44d4d326a3488cb57da87b3af42704b390dacb648752a503f931af274d2f"} Sep 07 13:10:58.691043 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:10:58.690922 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="418e44d4d326a3488cb57da87b3af42704b390dacb648752a503f931af274d2f" Sep 07 13:11:00.224127 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.224091 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:00.300716 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.300683 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:00.466650 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.466620 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:00.757542 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.757516 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56"] Sep 07 13:11:00.757871 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.757859 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07c3c902-175e-405a-9711-63e7dcfc66eb" containerName="revoke-keys" Sep 07 13:11:00.757925 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.757872 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="07c3c902-175e-405a-9711-63e7dcfc66eb" containerName="revoke-keys" Sep 07 13:11:00.757962 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.757936 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="07c3c902-175e-405a-9711-63e7dcfc66eb" containerName="revoke-keys" Sep 07 13:11:00.761057 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.761038 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.763371 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.763348 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-953bf16f-openshift-default-dockercfg-74k7l\"" Sep 07 13:11:00.778725 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.778700 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56"] Sep 07 13:11:00.788944 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.788917 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:00.874385 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.874348 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-workload-socket\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.874385 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.874389 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-envoy\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.874615 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.874463 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-token\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.874615 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.874509 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-workload-certs\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.874615 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.874548 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-data\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.874615 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.874563 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-podinfo\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.874615 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.874596 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f66570-9bbe-4126-9bd3-2e043fac91c8-istiod-ca-cert\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.874815 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.874617 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-credential-socket\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.874815 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.874631 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wms4n\" (UniqueName: \"kubernetes.io/projected/98f66570-9bbe-4126-9bd3-2e043fac91c8-kube-api-access-wms4n\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.975846 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.975811 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-workload-certs\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.975846 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.975851 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-data\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.976148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.975870 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-podinfo\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.976148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.975911 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f66570-9bbe-4126-9bd3-2e043fac91c8-istiod-ca-cert\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.976148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.975940 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-credential-socket\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.976148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.975956 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wms4n\" (UniqueName: \"kubernetes.io/projected/98f66570-9bbe-4126-9bd3-2e043fac91c8-kube-api-access-wms4n\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.976148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.976015 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-workload-socket\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.976148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.976039 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-envoy\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.976148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.976106 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-token\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.976514 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.976370 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-credential-socket\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.976514 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.976399 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-data\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.976609 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.976519 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-workload-socket\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.976609 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.976525 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-workload-certs\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.976743 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.976721 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f66570-9bbe-4126-9bd3-2e043fac91c8-istiod-ca-cert\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.978519 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.978497 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-envoy\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.978658 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.978640 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-podinfo\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.985415 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.985388 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-token\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:00.985697 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:00.985680 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wms4n\" (UniqueName: \"kubernetes.io/projected/98f66570-9bbe-4126-9bd3-2e043fac91c8-kube-api-access-wms4n\") pod \"e2e-mt-953bf16f-openshift-default-596cbffd96-glh56\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:01.009567 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:01.009492 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:01.071244 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:01.071217 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:01.350581 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:01.350552 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56"] Sep 07 13:11:01.352692 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:11:01.352669 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f66570_9bbe_4126_9bd3_2e043fac91c8.slice/crio-fbd6f792201c22a43c026c4ce74711a76bcaa736c9ac4c91249545c32b0fe3ea WatchSource:0}: Error finding container fbd6f792201c22a43c026c4ce74711a76bcaa736c9ac4c91249545c32b0fe3ea: Status 404 returned error can't find the container with id fbd6f792201c22a43c026c4ce74711a76bcaa736c9ac4c91249545c32b0fe3ea Sep 07 13:11:01.355073 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:01.355032 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:01.355187 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:01.355107 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:01.355187 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:01.355152 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:01.709335 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:01.709224 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" event={"ID":"98f66570-9bbe-4126-9bd3-2e043fac91c8","Type":"ContainerStarted","Data":"d2b1184c8c672b3b8dcfebf0ac8f48258caf2c309dcf890aa2843fd5533bd393"} Sep 07 13:11:01.709335 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:01.709299 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" event={"ID":"98f66570-9bbe-4126-9bd3-2e043fac91c8","Type":"ContainerStarted","Data":"fbd6f792201c22a43c026c4ce74711a76bcaa736c9ac4c91249545c32b0fe3ea"} Sep 07 13:11:01.729755 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:01.729696 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" podStartSLOduration=1.7296773779999999 podStartE2EDuration="1.729677378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:11:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:11:01.729400515 +0000 UTC m=+1095.760150305" watchObservedRunningTime="2026-09-07 13:11:01.729677378 +0000 UTC m=+1095.760427151" Sep 07 13:11:02.072019 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:02.071984 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:02.076635 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:02.076608 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:02.714632 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:02.714600 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:02.715680 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:02.715663 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:03.640853 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:03.640813 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:03.760927 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:03.760893 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:04.932949 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:04.932913 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:05.045033 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:05.044950 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:05.150355 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:05.150301 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:05.634226 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:05.634190 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:05.733303 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:05.733268 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:05.905546 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:05.905440 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:06.050908 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.050873 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:06.071450 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.071420 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299"] Sep 07 13:11:06.071986 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.071936 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" podUID="9cf3a9c5-c74f-4a9b-8651-a31efff94c8d" containerName="payload-processing" containerID="cri-o://46d547b0b3ce1c92738705fa8bbd4b83d4d8c4e16d2e0f55161e965d4c5fb711" gracePeriod=30 Sep 07 13:11:06.084998 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.084950 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64"] Sep 07 13:11:06.085247 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.085218 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" podUID="b19d504b-b304-431e-ac80-3c4e316c8063" containerName="payload-pre-processing" containerID="cri-o://aef69b99baf106a92849d6c1e8724b5a9924dae557cffac0614d128e365b7981" gracePeriod=30 Sep 07 13:11:06.150378 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.150343 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:06.259657 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.259612 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:06.342928 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.342904 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" Sep 07 13:11:06.376016 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.375993 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" Sep 07 13:11:06.432006 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.431909 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:06.487769 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.487744 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948"] Sep 07 13:11:06.487769 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.487776 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-76f028-hx948"] Sep 07 13:11:06.531673 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.531643 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcvjh\" (UniqueName: \"kubernetes.io/projected/b19d504b-b304-431e-ac80-3c4e316c8063-kube-api-access-dcvjh\") pod \"b19d504b-b304-431e-ac80-3c4e316c8063\" (UID: \"b19d504b-b304-431e-ac80-3c4e316c8063\") " Sep 07 13:11:06.531832 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.531688 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgpwq\" (UniqueName: \"kubernetes.io/projected/9cf3a9c5-c74f-4a9b-8651-a31efff94c8d-kube-api-access-pgpwq\") pod \"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d\" (UID: \"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d\") " Sep 07 13:11:06.531832 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.531776 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cf3a9c5-c74f-4a9b-8651-a31efff94c8d-plugins-config-volume\") pod \"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d\" (UID: \"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d\") " Sep 07 13:11:06.531832 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.531801 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b19d504b-b304-431e-ac80-3c4e316c8063-plugins-config-volume\") pod \"b19d504b-b304-431e-ac80-3c4e316c8063\" (UID: \"b19d504b-b304-431e-ac80-3c4e316c8063\") " Sep 07 13:11:06.532216 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.532190 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b19d504b-b304-431e-ac80-3c4e316c8063-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b19d504b-b304-431e-ac80-3c4e316c8063" (UID: "b19d504b-b304-431e-ac80-3c4e316c8063"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:11:06.532309 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.532190 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cf3a9c5-c74f-4a9b-8651-a31efff94c8d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9cf3a9c5-c74f-4a9b-8651-a31efff94c8d" (UID: "9cf3a9c5-c74f-4a9b-8651-a31efff94c8d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:11:06.533940 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.533917 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cf3a9c5-c74f-4a9b-8651-a31efff94c8d-kube-api-access-pgpwq" (OuterVolumeSpecName: "kube-api-access-pgpwq") pod "9cf3a9c5-c74f-4a9b-8651-a31efff94c8d" (UID: "9cf3a9c5-c74f-4a9b-8651-a31efff94c8d"). InnerVolumeSpecName "kube-api-access-pgpwq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:11:06.534068 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.533953 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b19d504b-b304-431e-ac80-3c4e316c8063-kube-api-access-dcvjh" (OuterVolumeSpecName: "kube-api-access-dcvjh") pod "b19d504b-b304-431e-ac80-3c4e316c8063" (UID: "b19d504b-b304-431e-ac80-3c4e316c8063"). InnerVolumeSpecName "kube-api-access-dcvjh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:11:06.633197 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.633155 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgpwq\" (UniqueName: \"kubernetes.io/projected/9cf3a9c5-c74f-4a9b-8651-a31efff94c8d-kube-api-access-pgpwq\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:11:06.633197 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.633192 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cf3a9c5-c74f-4a9b-8651-a31efff94c8d-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:11:06.633197 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.633206 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b19d504b-b304-431e-ac80-3c4e316c8063-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:11:06.633472 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.633219 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcvjh\" (UniqueName: \"kubernetes.io/projected/b19d504b-b304-431e-ac80-3c4e316c8063-kube-api-access-dcvjh\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:11:06.735474 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.735372 2576 generic.go:356] "Generic (PLEG): container finished" podID="b19d504b-b304-431e-ac80-3c4e316c8063" containerID="aef69b99baf106a92849d6c1e8724b5a9924dae557cffac0614d128e365b7981" exitCode=0 Sep 07 13:11:06.735474 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.735421 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" Sep 07 13:11:06.735474 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.735462 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" event={"ID":"b19d504b-b304-431e-ac80-3c4e316c8063","Type":"ContainerDied","Data":"aef69b99baf106a92849d6c1e8724b5a9924dae557cffac0614d128e365b7981"} Sep 07 13:11:06.735736 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.735525 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64" event={"ID":"b19d504b-b304-431e-ac80-3c4e316c8063","Type":"ContainerDied","Data":"6e7b775e3575025f0a3dcf422ab4f326db84798f558deb507cb8bc02f15a5210"} Sep 07 13:11:06.735736 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.735564 2576 scope.go:117] "RemoveContainer" containerID="aef69b99baf106a92849d6c1e8724b5a9924dae557cffac0614d128e365b7981" Sep 07 13:11:06.736853 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.736826 2576 generic.go:356] "Generic (PLEG): container finished" podID="9cf3a9c5-c74f-4a9b-8651-a31efff94c8d" containerID="46d547b0b3ce1c92738705fa8bbd4b83d4d8c4e16d2e0f55161e965d4c5fb711" exitCode=0 Sep 07 13:11:06.736924 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.736869 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" Sep 07 13:11:06.736924 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.736903 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" event={"ID":"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d","Type":"ContainerDied","Data":"46d547b0b3ce1c92738705fa8bbd4b83d4d8c4e16d2e0f55161e965d4c5fb711"} Sep 07 13:11:06.737031 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.736939 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299" event={"ID":"9cf3a9c5-c74f-4a9b-8651-a31efff94c8d","Type":"ContainerDied","Data":"03f6d752585cc814da8494f598092dc58046115d309e689a75ae696aba7dfa8c"} Sep 07 13:11:06.747600 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.747573 2576 scope.go:117] "RemoveContainer" containerID="aef69b99baf106a92849d6c1e8724b5a9924dae557cffac0614d128e365b7981" Sep 07 13:11:06.748072 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:11:06.748047 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aef69b99baf106a92849d6c1e8724b5a9924dae557cffac0614d128e365b7981\": container with ID starting with aef69b99baf106a92849d6c1e8724b5a9924dae557cffac0614d128e365b7981 not found: ID does not exist" containerID="aef69b99baf106a92849d6c1e8724b5a9924dae557cffac0614d128e365b7981" Sep 07 13:11:06.748176 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.748083 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aef69b99baf106a92849d6c1e8724b5a9924dae557cffac0614d128e365b7981"} err="failed to get container status \"aef69b99baf106a92849d6c1e8724b5a9924dae557cffac0614d128e365b7981\": rpc error: code = NotFound desc = could not find container \"aef69b99baf106a92849d6c1e8724b5a9924dae557cffac0614d128e365b7981\": container with ID starting with aef69b99baf106a92849d6c1e8724b5a9924dae557cffac0614d128e365b7981 not found: ID does not exist" Sep 07 13:11:06.748176 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.748108 2576 scope.go:117] "RemoveContainer" containerID="46d547b0b3ce1c92738705fa8bbd4b83d4d8c4e16d2e0f55161e965d4c5fb711" Sep 07 13:11:06.757030 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.757014 2576 scope.go:117] "RemoveContainer" containerID="46d547b0b3ce1c92738705fa8bbd4b83d4d8c4e16d2e0f55161e965d4c5fb711" Sep 07 13:11:06.757305 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:11:06.757286 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46d547b0b3ce1c92738705fa8bbd4b83d4d8c4e16d2e0f55161e965d4c5fb711\": container with ID starting with 46d547b0b3ce1c92738705fa8bbd4b83d4d8c4e16d2e0f55161e965d4c5fb711 not found: ID does not exist" containerID="46d547b0b3ce1c92738705fa8bbd4b83d4d8c4e16d2e0f55161e965d4c5fb711" Sep 07 13:11:06.757382 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.757314 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46d547b0b3ce1c92738705fa8bbd4b83d4d8c4e16d2e0f55161e965d4c5fb711"} err="failed to get container status \"46d547b0b3ce1c92738705fa8bbd4b83d4d8c4e16d2e0f55161e965d4c5fb711\": rpc error: code = NotFound desc = could not find container \"46d547b0b3ce1c92738705fa8bbd4b83d4d8c4e16d2e0f55161e965d4c5fb711\": container with ID starting with 46d547b0b3ce1c92738705fa8bbd4b83d4d8c4e16d2e0f55161e965d4c5fb711 not found: ID does not exist" Sep 07 13:11:06.767853 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.767834 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64"] Sep 07 13:11:06.770907 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.770888 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-76f028-9c448d77d-ggv64"] Sep 07 13:11:06.790307 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.790277 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:06.791466 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.791447 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299"] Sep 07 13:11:06.795425 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:06.795404 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-76f028-7cdc8c84c7-xj299"] Sep 07 13:11:08.482803 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:08.482773 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07c3c902-175e-405a-9711-63e7dcfc66eb" path="/var/lib/kubelet/pods/07c3c902-175e-405a-9711-63e7dcfc66eb/volumes" Sep 07 13:11:08.483197 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:08.483166 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cf3a9c5-c74f-4a9b-8651-a31efff94c8d" path="/var/lib/kubelet/pods/9cf3a9c5-c74f-4a9b-8651-a31efff94c8d/volumes" Sep 07 13:11:08.483587 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:08.483570 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b19d504b-b304-431e-ac80-3c4e316c8063" path="/var/lib/kubelet/pods/b19d504b-b304-431e-ac80-3c4e316c8063/volumes" Sep 07 13:11:13.732959 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:13.732925 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:13.806155 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:13.806122 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:13.996550 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:13.996422 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:14.572957 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.572915 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:14.873561 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.873471 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm"] Sep 07 13:11:14.874751 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.874727 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf3a9c5-c74f-4a9b-8651-a31efff94c8d" containerName="payload-processing" Sep 07 13:11:14.874922 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.874908 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf3a9c5-c74f-4a9b-8651-a31efff94c8d" containerName="payload-processing" Sep 07 13:11:14.875087 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.875074 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b19d504b-b304-431e-ac80-3c4e316c8063" containerName="payload-pre-processing" Sep 07 13:11:14.875209 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.875198 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b19d504b-b304-431e-ac80-3c4e316c8063" containerName="payload-pre-processing" Sep 07 13:11:14.875426 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.875413 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b19d504b-b304-431e-ac80-3c4e316c8063" containerName="payload-pre-processing" Sep 07 13:11:14.875558 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.875546 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cf3a9c5-c74f-4a9b-8651-a31efff94c8d" containerName="payload-processing" Sep 07 13:11:14.886444 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.886422 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:14.899447 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.899169 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-aa813e-openshift-default-dockercfg-cbzg4\"" Sep 07 13:11:14.903528 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.902867 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm"] Sep 07 13:11:14.904769 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.904681 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:14.913080 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.913052 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-podinfo\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:14.913180 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.913100 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-token\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:14.913180 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.913154 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2977116-3833-4166-8fa6-b4d200bfeeb2-istiod-ca-cert\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:14.913290 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.913265 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46ctd\" (UniqueName: \"kubernetes.io/projected/a2977116-3833-4166-8fa6-b4d200bfeeb2-kube-api-access-46ctd\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:14.913398 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.913329 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-credential-socket\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:14.913398 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.913366 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-envoy\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:14.913398 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.913391 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-data\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:14.913509 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.913437 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-workload-socket\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:14.913509 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:14.913462 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-workload-certs\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.014956 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.014919 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-credential-socket\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.014956 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.014952 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-envoy\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.015229 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.014997 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-data\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.015229 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.015044 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-workload-socket\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.015229 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.015071 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-workload-certs\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.015229 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.015162 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-podinfo\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.015229 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.015196 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-token\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.015229 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.015218 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2977116-3833-4166-8fa6-b4d200bfeeb2-istiod-ca-cert\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.015541 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.015253 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46ctd\" (UniqueName: \"kubernetes.io/projected/a2977116-3833-4166-8fa6-b4d200bfeeb2-kube-api-access-46ctd\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.015541 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.015471 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-workload-socket\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.015541 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.015524 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-data\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.015759 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.015562 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-workload-certs\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.015759 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.015746 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-credential-socket\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.016202 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.016180 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2977116-3833-4166-8fa6-b4d200bfeeb2-istiod-ca-cert\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.018368 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.018331 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-envoy\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.018591 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.018572 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-podinfo\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.031404 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.031378 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-token\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.033032 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.033011 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46ctd\" (UniqueName: \"kubernetes.io/projected/a2977116-3833-4166-8fa6-b4d200bfeeb2-kube-api-access-46ctd\") pod \"e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.081742 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.081661 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:15.184834 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.184733 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:15.203821 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.203792 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:15.360065 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.360027 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:15.498257 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.496209 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm"] Sep 07 13:11:15.499133 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:11:15.499105 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2977116_3833_4166_8fa6_b4d200bfeeb2.slice/crio-858da910177339c8c7640de3e118e788582fada7105feef571bec1ee7cd6e969 WatchSource:0}: Error finding container 858da910177339c8c7640de3e118e788582fada7105feef571bec1ee7cd6e969: Status 404 returned error can't find the container with id 858da910177339c8c7640de3e118e788582fada7105feef571bec1ee7cd6e969 Sep 07 13:11:15.501323 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.501291 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:15.501409 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.501367 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:15.501409 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.501398 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:15.797623 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.797439 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" event={"ID":"a2977116-3833-4166-8fa6-b4d200bfeeb2","Type":"ContainerStarted","Data":"e228054be5075cec783b260c6db5180654e65359eb83fe743c56ea60447da0cc"} Sep 07 13:11:15.797623 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.797510 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" event={"ID":"a2977116-3833-4166-8fa6-b4d200bfeeb2","Type":"ContainerStarted","Data":"858da910177339c8c7640de3e118e788582fada7105feef571bec1ee7cd6e969"} Sep 07 13:11:15.827474 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:15.827428 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" podStartSLOduration=1.827413856 podStartE2EDuration="1.827413856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:11:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:11:15.825528484 +0000 UTC m=+1109.856278283" watchObservedRunningTime="2026-09-07 13:11:15.827413856 +0000 UTC m=+1109.858163626" Sep 07 13:11:16.205181 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:16.205081 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:16.217532 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:16.217501 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:16.802048 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:16.802014 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:16.802988 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:16.802953 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:17.803773 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:17.803722 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:17.986825 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:17.986789 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:18.756148 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:18.756102 2576 patch_prober.go:28] interesting pod/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 07 13:11:18.756367 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:18.756168 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 07 13:11:18.965653 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:18.965614 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:19.066775 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.066731 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:19.170455 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.170415 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:19.652292 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.652256 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:19.751718 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.751621 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:19.820879 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.820792 2576 generic.go:356] "Generic (PLEG): container finished" podID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerID="d2b1184c8c672b3b8dcfebf0ac8f48258caf2c309dcf890aa2843fd5533bd393" exitCode=0 Sep 07 13:11:19.820879 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.820870 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" event={"ID":"98f66570-9bbe-4126-9bd3-2e043fac91c8","Type":"ContainerDied","Data":"d2b1184c8c672b3b8dcfebf0ac8f48258caf2c309dcf890aa2843fd5533bd393"} Sep 07 13:11:19.821309 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.821295 2576 scope.go:117] "RemoveContainer" containerID="d2b1184c8c672b3b8dcfebf0ac8f48258caf2c309dcf890aa2843fd5533bd393" Sep 07 13:11:19.822399 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.822375 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:19.822472 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.822459 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:19.822513 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.822495 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:19.847874 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.847839 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh"] Sep 07 13:11:19.852323 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.852302 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" Sep 07 13:11:19.854049 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.854031 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-aa813e-dockercfg-dpmxq\"" Sep 07 13:11:19.854133 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.854114 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-aa813e\"" Sep 07 13:11:19.873020 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.872989 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh"] Sep 07 13:11:19.958330 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.958284 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwjxn\" (UniqueName: \"kubernetes.io/projected/a4e744ff-e3da-480c-8144-8eab85262223-kube-api-access-lwjxn\") pod \"payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh\" (UID: \"a4e744ff-e3da-480c-8144-8eab85262223\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" Sep 07 13:11:19.958496 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:19.958369 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4e744ff-e3da-480c-8144-8eab85262223-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh\" (UID: \"a4e744ff-e3da-480c-8144-8eab85262223\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" Sep 07 13:11:20.058915 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:20.058884 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwjxn\" (UniqueName: \"kubernetes.io/projected/a4e744ff-e3da-480c-8144-8eab85262223-kube-api-access-lwjxn\") pod \"payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh\" (UID: \"a4e744ff-e3da-480c-8144-8eab85262223\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" Sep 07 13:11:20.059298 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:20.058926 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4e744ff-e3da-480c-8144-8eab85262223-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh\" (UID: \"a4e744ff-e3da-480c-8144-8eab85262223\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" Sep 07 13:11:20.059673 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:20.059652 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4e744ff-e3da-480c-8144-8eab85262223-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh\" (UID: \"a4e744ff-e3da-480c-8144-8eab85262223\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" Sep 07 13:11:20.067864 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:20.067839 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwjxn\" (UniqueName: \"kubernetes.io/projected/a4e744ff-e3da-480c-8144-8eab85262223-kube-api-access-lwjxn\") pod \"payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh\" (UID: \"a4e744ff-e3da-480c-8144-8eab85262223\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" Sep 07 13:11:20.175393 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:20.175293 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" Sep 07 13:11:20.457629 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:20.457598 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh"] Sep 07 13:11:20.459650 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:11:20.459625 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e744ff_e3da_480c_8144_8eab85262223.slice/crio-cc5712540f64ef274a3349edc7ae1caffc108cc1dcd53001f1e7df2f27038d28 WatchSource:0}: Error finding container cc5712540f64ef274a3349edc7ae1caffc108cc1dcd53001f1e7df2f27038d28: Status 404 returned error can't find the container with id cc5712540f64ef274a3349edc7ae1caffc108cc1dcd53001f1e7df2f27038d28 Sep 07 13:11:20.827820 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:20.827786 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" event={"ID":"98f66570-9bbe-4126-9bd3-2e043fac91c8","Type":"ContainerStarted","Data":"23bade2f116a21884fc90097223cf53ea5136979482d9f4a1c0ce4f96722976b"} Sep 07 13:11:20.829413 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:20.829387 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" event={"ID":"a4e744ff-e3da-480c-8144-8eab85262223","Type":"ContainerStarted","Data":"6c4d030476e563df9fb6671ac713271716c879a31a01bbcb3b384dd38cc14082"} Sep 07 13:11:20.829543 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:20.829432 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" event={"ID":"a4e744ff-e3da-480c-8144-8eab85262223","Type":"ContainerStarted","Data":"cc5712540f64ef274a3349edc7ae1caffc108cc1dcd53001f1e7df2f27038d28"} Sep 07 13:11:20.829543 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:20.829457 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" Sep 07 13:11:21.074489 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:21.074456 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:21.075371 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:21.075321 2576 patch_prober.go:28] interesting pod/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Sep 07 13:11:21.075621 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:21.075578 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Sep 07 13:11:22.072561 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:22.072513 2576 patch_prober.go:28] interesting pod/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Sep 07 13:11:22.072774 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:22.072583 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Sep 07 13:11:23.072669 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:23.072622 2576 patch_prober.go:28] interesting pod/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Sep 07 13:11:23.073190 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:23.072691 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Sep 07 13:11:24.072562 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:24.072510 2576 patch_prober.go:28] interesting pod/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Sep 07 13:11:24.072797 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:24.072586 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Sep 07 13:11:25.072461 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:25.072422 2576 patch_prober.go:28] interesting pod/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Sep 07 13:11:25.073217 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:25.072777 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Sep 07 13:11:26.072479 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:26.072440 2576 patch_prober.go:28] interesting pod/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Sep 07 13:11:26.072794 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:26.072754 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Sep 07 13:11:26.112643 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:26.112606 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:27.081511 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:27.081469 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:27.082801 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:27.082772 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:27.086819 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:27.086792 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:11:27.106436 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:27.106381 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" podStartSLOduration=8.106365641 podStartE2EDuration="8.106365641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:11:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:11:20.873526896 +0000 UTC m=+1114.904276668" watchObservedRunningTime="2026-09-07 13:11:27.106365641 +0000 UTC m=+1121.137115411" Sep 07 13:11:28.602326 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:28.602276 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:29.275747 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:29.275701 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:29.376674 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:29.376614 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:29.721012 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:29.720953 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:31.803317 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:31.803283 2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 07 13:11:31.803776 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:31.803342 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 07 13:11:31.837124 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:31.837095 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" Sep 07 13:11:31.887716 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:31.887627 2576 generic.go:356] "Generic (PLEG): container finished" podID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerID="e228054be5075cec783b260c6db5180654e65359eb83fe743c56ea60447da0cc" exitCode=0 Sep 07 13:11:31.887884 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:31.887704 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" event={"ID":"a2977116-3833-4166-8fa6-b4d200bfeeb2","Type":"ContainerDied","Data":"e228054be5075cec783b260c6db5180654e65359eb83fe743c56ea60447da0cc"} Sep 07 13:11:31.888183 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:31.888166 2576 scope.go:117] "RemoveContainer" containerID="e228054be5075cec783b260c6db5180654e65359eb83fe743c56ea60447da0cc" Sep 07 13:11:31.889213 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:31.889183 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:31.889331 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:31.889264 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:31.889331 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:31.889313 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:32.896414 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:32.896378 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" event={"ID":"a2977116-3833-4166-8fa6-b4d200bfeeb2","Type":"ContainerStarted","Data":"81adc1826f0ba736744570e66c10794a37d034f6ac113f376f124f33f5124e03"} Sep 07 13:11:33.204749 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:33.204643 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:33.204953 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:33.204925 2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 07 13:11:33.205072 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:33.205022 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 07 13:11:34.205078 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:34.205044 2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 07 13:11:34.205474 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:34.205111 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 07 13:11:35.204612 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:35.204571 2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 07 13:11:35.204807 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:35.204642 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 07 13:11:36.204998 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:36.204922 2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 07 13:11:36.205403 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:36.205030 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 07 13:11:36.906267 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:36.906231 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:36.985891 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:36.985851 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:37.204303 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:37.204210 2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 07 13:11:37.204303 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:37.204277 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 07 13:11:38.204589 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:38.204551 2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 07 13:11:38.205077 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:38.204651 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 07 13:11:39.207816 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:39.207786 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:39.208215 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:39.208169 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:39.209023 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:39.209000 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:11:40.183381 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:40.183351 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:42.765694 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:42.765651 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:43.071765 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.071726 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:43.356643 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.356566 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx"] Sep 07 13:11:43.359910 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.359885 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.361704 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.361672 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-ae93b0-openshift-default-dockercfg-vvw84\"" Sep 07 13:11:43.376570 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.376545 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx"] Sep 07 13:11:43.387927 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.387898 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:43.496084 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.496050 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/139ce760-9860-4570-8f58-0e31acd54c2d-istio-token\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.496084 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.496083 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-workload-certs\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.496321 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.496101 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/139ce760-9860-4570-8f58-0e31acd54c2d-istio-podinfo\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.496321 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.496174 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-istio-data\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.496321 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.496249 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/139ce760-9860-4570-8f58-0e31acd54c2d-istiod-ca-cert\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.496321 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.496301 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-workload-socket\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.496532 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.496344 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6vvm\" (UniqueName: \"kubernetes.io/projected/139ce760-9860-4570-8f58-0e31acd54c2d-kube-api-access-p6vvm\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.496532 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.496419 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-istio-envoy\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.496532 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.496453 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-credential-socket\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.597356 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.597322 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/139ce760-9860-4570-8f58-0e31acd54c2d-istiod-ca-cert\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.597535 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.597361 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-workload-socket\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.597535 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.597445 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6vvm\" (UniqueName: \"kubernetes.io/projected/139ce760-9860-4570-8f58-0e31acd54c2d-kube-api-access-p6vvm\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.597535 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.597498 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-istio-envoy\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.597535 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.597521 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-credential-socket\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.597770 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.597739 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/139ce760-9860-4570-8f58-0e31acd54c2d-istio-token\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.597833 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.597775 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-workload-certs\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.597833 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.597802 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-workload-socket\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.598026 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.598006 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-credential-socket\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.598105 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.598070 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/139ce760-9860-4570-8f58-0e31acd54c2d-istio-podinfo\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.598155 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.598096 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-workload-certs\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.598155 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.598134 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-istio-data\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.598241 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.598199 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/139ce760-9860-4570-8f58-0e31acd54c2d-istiod-ca-cert\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.598343 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.598328 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-istio-data\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.600005 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.599954 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-istio-envoy\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.600639 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.600617 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/139ce760-9860-4570-8f58-0e31acd54c2d-istio-podinfo\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.609877 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.609823 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6vvm\" (UniqueName: \"kubernetes.io/projected/139ce760-9860-4570-8f58-0e31acd54c2d-kube-api-access-p6vvm\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.609877 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.609839 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/139ce760-9860-4570-8f58-0e31acd54c2d-istio-token\") pod \"e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.669812 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.669782 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:43.962450 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.962418 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx"] Sep 07 13:11:43.964856 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:11:43.964826 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod139ce760_9860_4570_8f58_0e31acd54c2d.slice/crio-b75f5984485871dfb3195d588c5162c1ace4c7ffb58cedface80b88585dffdfb WatchSource:0}: Error finding container b75f5984485871dfb3195d588c5162c1ace4c7ffb58cedface80b88585dffdfb: Status 404 returned error can't find the container with id b75f5984485871dfb3195d588c5162c1ace4c7ffb58cedface80b88585dffdfb Sep 07 13:11:43.967014 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.966962 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:43.967113 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.967062 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:43.967113 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:43.967105 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:11:44.320058 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:44.320023 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:44.970664 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:44.970615 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" event={"ID":"139ce760-9860-4570-8f58-0e31acd54c2d","Type":"ContainerStarted","Data":"37d989e9ac72024cac922416f1c8e26365ba80737cbc4c1ac0415060c2f6c975"} Sep 07 13:11:44.971054 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:44.970674 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" event={"ID":"139ce760-9860-4570-8f58-0e31acd54c2d","Type":"ContainerStarted","Data":"b75f5984485871dfb3195d588c5162c1ace4c7ffb58cedface80b88585dffdfb"} Sep 07 13:11:45.003397 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:45.003348 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" podStartSLOduration=2.003333953 podStartE2EDuration="2.003333953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:11:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:11:45.001128738 +0000 UTC m=+1139.031878519" watchObservedRunningTime="2026-09-07 13:11:45.003333953 +0000 UTC m=+1139.034083724" Sep 07 13:11:45.670467 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:45.670433 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:45.675323 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:45.675301 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:45.975992 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:45.975871 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:45.976806 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:45.976790 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:11:46.994900 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:46.994801 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:47.112830 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:47.112794 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:47.595157 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:47.595121 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:49.341175 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:49.341139 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:49.446662 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:49.446621 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:49.577731 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:49.577694 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:50.170338 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.170303 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:50.264638 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.264604 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:50.431486 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.431399 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd"] Sep 07 13:11:50.436384 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.436359 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" Sep 07 13:11:50.439259 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.439230 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-ae93b0-dockercfg-2dqcw\"" Sep 07 13:11:50.439658 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.439635 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-ae93b0\"" Sep 07 13:11:50.447706 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.447388 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd"] Sep 07 13:11:50.561087 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.561051 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12086bc5-5974-48b7-bca6-77757f20f5e9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd\" (UID: \"12086bc5-5974-48b7-bca6-77757f20f5e9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" Sep 07 13:11:50.561278 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.561202 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ghd6\" (UniqueName: \"kubernetes.io/projected/12086bc5-5974-48b7-bca6-77757f20f5e9-kube-api-access-5ghd6\") pod \"payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd\" (UID: \"12086bc5-5974-48b7-bca6-77757f20f5e9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" Sep 07 13:11:50.653305 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.653270 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:50.662368 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.662339 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ghd6\" (UniqueName: \"kubernetes.io/projected/12086bc5-5974-48b7-bca6-77757f20f5e9-kube-api-access-5ghd6\") pod \"payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd\" (UID: \"12086bc5-5974-48b7-bca6-77757f20f5e9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" Sep 07 13:11:50.662491 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.662412 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12086bc5-5974-48b7-bca6-77757f20f5e9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd\" (UID: \"12086bc5-5974-48b7-bca6-77757f20f5e9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" Sep 07 13:11:50.663065 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.663046 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12086bc5-5974-48b7-bca6-77757f20f5e9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd\" (UID: \"12086bc5-5974-48b7-bca6-77757f20f5e9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" Sep 07 13:11:50.676284 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.676257 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ghd6\" (UniqueName: \"kubernetes.io/projected/12086bc5-5974-48b7-bca6-77757f20f5e9-kube-api-access-5ghd6\") pod \"payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd\" (UID: \"12086bc5-5974-48b7-bca6-77757f20f5e9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" Sep 07 13:11:50.752084 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.751993 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" Sep 07 13:11:50.778155 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.778116 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:50.875929 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.875892 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:50.957900 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:50.957866 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:51.034798 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:51.034764 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd"] Sep 07 13:11:51.287317 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:51.287221 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:52.011964 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:52.011925 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" event={"ID":"12086bc5-5974-48b7-bca6-77757f20f5e9","Type":"ContainerStarted","Data":"5a17983b86cc79ec432d507ac0de07c592ebcbdd8121de41753d4a4a2cea75d3"} Sep 07 13:11:52.012374 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:52.012030 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" event={"ID":"12086bc5-5974-48b7-bca6-77757f20f5e9","Type":"ContainerStarted","Data":"f7003ca4a5ea5c003c1a77a1018be742600913319e59e4f7095a873305789d22"} Sep 07 13:11:52.012374 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:52.012063 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" Sep 07 13:11:52.035327 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:52.035276 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" podStartSLOduration=2.035259004 podStartE2EDuration="2.035259004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:11:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:11:52.033903145 +0000 UTC m=+1146.064652937" watchObservedRunningTime="2026-09-07 13:11:52.035259004 +0000 UTC m=+1146.066008819" Sep 07 13:11:58.658434 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:58.658393 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:58.747201 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:58.747165 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:58.927440 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:58.927354 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56"] Sep 07 13:11:58.927584 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:58.927560 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" containerID="cri-o://23bade2f116a21884fc90097223cf53ea5136979482d9f4a1c0ce4f96722976b" gracePeriod=30 Sep 07 13:11:59.856025 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:59.855959 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:11:59.962998 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:11:59.962931 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:02.017159 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:02.017104 2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 07 13:12:02.017682 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:02.017197 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 07 13:12:03.021156 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:03.021124 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" Sep 07 13:12:04.080799 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.080766 2576 generic.go:356] "Generic (PLEG): container finished" podID="139ce760-9860-4570-8f58-0e31acd54c2d" containerID="37d989e9ac72024cac922416f1c8e26365ba80737cbc4c1ac0415060c2f6c975" exitCode=0 Sep 07 13:12:04.081330 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.080845 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" event={"ID":"139ce760-9860-4570-8f58-0e31acd54c2d","Type":"ContainerDied","Data":"37d989e9ac72024cac922416f1c8e26365ba80737cbc4c1ac0415060c2f6c975"} Sep 07 13:12:04.081418 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.081383 2576 scope.go:117] "RemoveContainer" containerID="37d989e9ac72024cac922416f1c8e26365ba80737cbc4c1ac0415060c2f6c975" Sep 07 13:12:04.082709 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.082676 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:12:04.082821 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.082755 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:12:04.082821 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.082788 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:12:04.083185 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.083164 2576 generic.go:356] "Generic (PLEG): container finished" podID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerID="23bade2f116a21884fc90097223cf53ea5136979482d9f4a1c0ce4f96722976b" exitCode=0 Sep 07 13:12:04.083275 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.083233 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" event={"ID":"98f66570-9bbe-4126-9bd3-2e043fac91c8","Type":"ContainerDied","Data":"23bade2f116a21884fc90097223cf53ea5136979482d9f4a1c0ce4f96722976b"} Sep 07 13:12:04.083324 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.083286 2576 scope.go:117] "RemoveContainer" containerID="d2b1184c8c672b3b8dcfebf0ac8f48258caf2c309dcf890aa2843fd5533bd393" Sep 07 13:12:04.199389 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.199322 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:12:04.292470 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.292436 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-envoy\") pod \"98f66570-9bbe-4126-9bd3-2e043fac91c8\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " Sep 07 13:12:04.292654 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.292485 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-credential-socket\") pod \"98f66570-9bbe-4126-9bd3-2e043fac91c8\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " Sep 07 13:12:04.292654 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.292534 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-podinfo\") pod \"98f66570-9bbe-4126-9bd3-2e043fac91c8\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " Sep 07 13:12:04.292654 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.292559 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f66570-9bbe-4126-9bd3-2e043fac91c8-istiod-ca-cert\") pod \"98f66570-9bbe-4126-9bd3-2e043fac91c8\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " Sep 07 13:12:04.292654 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.292599 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-workload-socket\") pod \"98f66570-9bbe-4126-9bd3-2e043fac91c8\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " Sep 07 13:12:04.292654 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.292620 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-data\") pod \"98f66570-9bbe-4126-9bd3-2e043fac91c8\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " Sep 07 13:12:04.292927 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.292820 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "98f66570-9bbe-4126-9bd3-2e043fac91c8" (UID: "98f66570-9bbe-4126-9bd3-2e043fac91c8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:12:04.292927 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.292878 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wms4n\" (UniqueName: \"kubernetes.io/projected/98f66570-9bbe-4126-9bd3-2e043fac91c8-kube-api-access-wms4n\") pod \"98f66570-9bbe-4126-9bd3-2e043fac91c8\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " Sep 07 13:12:04.293056 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.292924 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-token\") pod \"98f66570-9bbe-4126-9bd3-2e043fac91c8\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " Sep 07 13:12:04.293056 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.292955 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-data" (OuterVolumeSpecName: "istio-data") pod "98f66570-9bbe-4126-9bd3-2e043fac91c8" (UID: "98f66570-9bbe-4126-9bd3-2e043fac91c8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:12:04.293056 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.292962 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-workload-certs\") pod \"98f66570-9bbe-4126-9bd3-2e043fac91c8\" (UID: \"98f66570-9bbe-4126-9bd3-2e043fac91c8\") " Sep 07 13:12:04.293056 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.293044 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98f66570-9bbe-4126-9bd3-2e043fac91c8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "98f66570-9bbe-4126-9bd3-2e043fac91c8" (UID: "98f66570-9bbe-4126-9bd3-2e043fac91c8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:12:04.293262 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.293137 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "98f66570-9bbe-4126-9bd3-2e043fac91c8" (UID: "98f66570-9bbe-4126-9bd3-2e043fac91c8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:12:04.293337 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.293316 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "98f66570-9bbe-4126-9bd3-2e043fac91c8" (UID: "98f66570-9bbe-4126-9bd3-2e043fac91c8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:12:04.293420 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.293393 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-credential-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:12:04.293420 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.293411 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f66570-9bbe-4126-9bd3-2e043fac91c8-istiod-ca-cert\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:12:04.293520 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.293426 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-workload-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:12:04.293520 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.293439 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-data\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:12:04.295293 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.295262 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "98f66570-9bbe-4126-9bd3-2e043fac91c8" (UID: "98f66570-9bbe-4126-9bd3-2e043fac91c8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:12:04.295621 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.295589 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f66570-9bbe-4126-9bd3-2e043fac91c8-kube-api-access-wms4n" (OuterVolumeSpecName: "kube-api-access-wms4n") pod "98f66570-9bbe-4126-9bd3-2e043fac91c8" (UID: "98f66570-9bbe-4126-9bd3-2e043fac91c8"). InnerVolumeSpecName "kube-api-access-wms4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:12:04.295786 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.295767 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-token" (OuterVolumeSpecName: "istio-token") pod "98f66570-9bbe-4126-9bd3-2e043fac91c8" (UID: "98f66570-9bbe-4126-9bd3-2e043fac91c8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:12:04.295858 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.295790 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "98f66570-9bbe-4126-9bd3-2e043fac91c8" (UID: "98f66570-9bbe-4126-9bd3-2e043fac91c8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 13:12:04.394629 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.394596 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-podinfo\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:12:04.394629 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.394627 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wms4n\" (UniqueName: \"kubernetes.io/projected/98f66570-9bbe-4126-9bd3-2e043fac91c8-kube-api-access-wms4n\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:12:04.394833 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.394640 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-token\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:12:04.394833 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.394652 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-workload-certs\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:12:04.394833 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:04.394662 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f66570-9bbe-4126-9bd3-2e043fac91c8-istio-envoy\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:12:05.094755 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:05.094720 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" event={"ID":"139ce760-9860-4570-8f58-0e31acd54c2d","Type":"ContainerStarted","Data":"cd2888d827055b44df378cb268b626cc0cb9bb8e2c4042d2aa25c498f0e91e86"} Sep 07 13:12:05.096199 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:05.096178 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" Sep 07 13:12:05.096358 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:05.096210 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56" event={"ID":"98f66570-9bbe-4126-9bd3-2e043fac91c8","Type":"ContainerDied","Data":"fbd6f792201c22a43c026c4ce74711a76bcaa736c9ac4c91249545c32b0fe3ea"} Sep 07 13:12:05.096358 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:05.096256 2576 scope.go:117] "RemoveContainer" containerID="23bade2f116a21884fc90097223cf53ea5136979482d9f4a1c0ce4f96722976b" Sep 07 13:12:05.139731 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:05.139697 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56"] Sep 07 13:12:05.152738 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:05.152707 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-953bf16f-openshift-default-596cbffd96-glh56"] Sep 07 13:12:05.670987 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:05.670926 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:12:05.671208 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:05.671083 2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Sep 07 13:12:05.671208 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:05.671145 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Sep 07 13:12:06.456200 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:06.456159 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:06.490090 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:06.490054 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" path="/var/lib/kubelet/pods/98f66570-9bbe-4126-9bd3-2e043fac91c8/volumes" Sep 07 13:12:06.555690 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:06.555635 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:06.658948 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:06.658918 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:06.670613 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:06.670575 2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Sep 07 13:12:06.670796 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:06.670639 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Sep 07 13:12:06.817821 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:06.817781 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:07.670811 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:07.670700 2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Sep 07 13:12:07.670811 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:07.670788 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Sep 07 13:12:08.671060 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:08.671015 2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Sep 07 13:12:08.671577 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:08.671103 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Sep 07 13:12:09.670723 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:09.670687 2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Sep 07 13:12:09.671072 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:09.671040 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Sep 07 13:12:10.673902 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:10.673867 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:12:10.674319 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:10.674194 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:12:10.674951 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:10.674935 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:12:11.904531 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:11.904496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:13.799882 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:13.799851 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:13.863208 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:13.863177 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:14.701332 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:14.701302 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:15.055276 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:15.055231 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:15.545918 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:15.545882 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:18.788360 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:18.788324 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:18.883026 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:18.882960 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:19.488037 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:19.487994 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:19.590695 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:19.590658 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:19.767995 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:19.767893 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:20.126869 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:20.126838 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:20.225241 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:20.225207 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:29.316507 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:29.316471 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:43.298584 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:43.298544 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:43.645757 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:43.645665 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:43.981009 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:43.980909 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw"] Sep 07 13:12:43.981299 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:43.981286 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" Sep 07 13:12:43.981350 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:43.981300 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" Sep 07 13:12:43.981350 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:43.981320 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" Sep 07 13:12:43.981350 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:43.981325 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" Sep 07 13:12:43.981460 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:43.981390 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" Sep 07 13:12:43.981460 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:43.981399 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f66570-9bbe-4126-9bd3-2e043fac91c8" containerName="istio-proxy" Sep 07 13:12:43.984389 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:43.984367 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:43.986362 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:43.986340 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-45d721-openshift-default-dockercfg-thpzx\"" Sep 07 13:12:44.001876 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.001857 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw"] Sep 07 13:12:44.021227 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.021191 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:44.059854 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.059825 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c45411e-d304-461f-aef1-f6ac915b9a8f-istiod-ca-cert\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.059854 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.059855 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vh9n\" (UniqueName: \"kubernetes.io/projected/3c45411e-d304-461f-aef1-f6ac915b9a8f-kube-api-access-2vh9n\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.060137 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.059885 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-envoy\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.060137 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.059933 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-workload-socket\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.060137 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.059990 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-token\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.060137 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.060058 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-credential-socket\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.060137 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.060114 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-data\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.060351 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.060167 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-workload-certs\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.060351 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.060202 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-podinfo\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.161402 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.161358 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c45411e-d304-461f-aef1-f6ac915b9a8f-istiod-ca-cert\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.161610 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.161414 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vh9n\" (UniqueName: \"kubernetes.io/projected/3c45411e-d304-461f-aef1-f6ac915b9a8f-kube-api-access-2vh9n\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.161610 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.161456 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-envoy\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.161610 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.161481 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-workload-socket\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.161610 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.161499 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-token\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.161610 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.161525 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-credential-socket\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.161610 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.161554 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-data\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.162063 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.161874 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-workload-certs\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.162063 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.161911 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-data\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.162063 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.161917 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-podinfo\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.162063 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.161953 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-workload-socket\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.162063 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.162021 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-credential-socket\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.162314 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.162235 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c45411e-d304-461f-aef1-f6ac915b9a8f-istiod-ca-cert\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.162314 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.162250 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-workload-certs\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.163909 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.163886 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-envoy\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.164348 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.164331 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-podinfo\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.170394 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.170370 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-token\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.170556 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.170538 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vh9n\" (UniqueName: \"kubernetes.io/projected/3c45411e-d304-461f-aef1-f6ac915b9a8f-kube-api-access-2vh9n\") pod \"e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.294460 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.294428 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:44.572538 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.572514 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw"] Sep 07 13:12:44.575080 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:12:44.575042 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c45411e_d304_461f_aef1_f6ac915b9a8f.slice/crio-6bdd4d9416204e7b08e75c1b80b926b3d90222abb00e48aa42047e331cea043a WatchSource:0}: Error finding container 6bdd4d9416204e7b08e75c1b80b926b3d90222abb00e48aa42047e331cea043a: Status 404 returned error can't find the container with id 6bdd4d9416204e7b08e75c1b80b926b3d90222abb00e48aa42047e331cea043a Sep 07 13:12:44.577327 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.577293 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:12:44.577454 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.577360 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:12:44.577454 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.577400 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:12:44.686264 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:44.686229 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:45.320624 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:45.320588 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" event={"ID":"3c45411e-d304-461f-aef1-f6ac915b9a8f","Type":"ContainerStarted","Data":"aa5c4ed0f10dc1cd75db15114b8fb70398ceff0d5fe6a6e38b985fd33adb4735"} Sep 07 13:12:45.320800 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:45.320684 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" event={"ID":"3c45411e-d304-461f-aef1-f6ac915b9a8f","Type":"ContainerStarted","Data":"6bdd4d9416204e7b08e75c1b80b926b3d90222abb00e48aa42047e331cea043a"} Sep 07 13:12:45.341747 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:45.341704 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" podStartSLOduration=2.341687883 podStartE2EDuration="2.341687883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:12:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:12:45.33970214 +0000 UTC m=+1199.370451913" watchObservedRunningTime="2026-09-07 13:12:45.341687883 +0000 UTC m=+1199.372437654" Sep 07 13:12:46.295472 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:46.295441 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:46.300950 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:46.300919 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:46.326230 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:46.326205 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:46.327008 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:46.326962 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:12:46.482854 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:46.482828 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 13:12:46.483074 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:46.483044 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 13:12:47.281930 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:47.281897 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:47.395712 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:47.395677 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:48.521332 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:48.521300 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:48.619659 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:48.619619 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:49.003663 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.003628 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:49.106356 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.106323 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:49.129227 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.129200 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5"] Sep 07 13:12:49.133993 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.133958 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" Sep 07 13:12:49.135690 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.135667 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-45d721-dockercfg-dk6vw\"" Sep 07 13:12:49.135817 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.135670 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-45d721\"" Sep 07 13:12:49.143425 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.143403 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5"] Sep 07 13:12:49.201742 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.201715 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e135966-53cb-4eca-917d-efcba7c2595f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5\" (UID: \"8e135966-53cb-4eca-917d-efcba7c2595f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" Sep 07 13:12:49.201902 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.201757 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs7t4\" (UniqueName: \"kubernetes.io/projected/8e135966-53cb-4eca-917d-efcba7c2595f-kube-api-access-qs7t4\") pod \"payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5\" (UID: \"8e135966-53cb-4eca-917d-efcba7c2595f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" Sep 07 13:12:49.202526 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.202499 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:12:49.303204 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.303176 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e135966-53cb-4eca-917d-efcba7c2595f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5\" (UID: \"8e135966-53cb-4eca-917d-efcba7c2595f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" Sep 07 13:12:49.303358 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.303224 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs7t4\" (UniqueName: \"kubernetes.io/projected/8e135966-53cb-4eca-917d-efcba7c2595f-kube-api-access-qs7t4\") pod \"payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5\" (UID: \"8e135966-53cb-4eca-917d-efcba7c2595f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" Sep 07 13:12:49.303911 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.303891 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e135966-53cb-4eca-917d-efcba7c2595f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5\" (UID: \"8e135966-53cb-4eca-917d-efcba7c2595f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" Sep 07 13:12:49.311889 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.311864 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs7t4\" (UniqueName: \"kubernetes.io/projected/8e135966-53cb-4eca-917d-efcba7c2595f-kube-api-access-qs7t4\") pod \"payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5\" (UID: \"8e135966-53cb-4eca-917d-efcba7c2595f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" Sep 07 13:12:49.444852 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.444818 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" Sep 07 13:12:49.725559 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:49.725534 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5"] Sep 07 13:12:49.727579 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:12:49.727536 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e135966_53cb_4eca_917d_efcba7c2595f.slice/crio-f718395397b0f754809433a4666b2df69aa89da6e3a6b6bb30e5e6bbadc583a9 WatchSource:0}: Error finding container f718395397b0f754809433a4666b2df69aa89da6e3a6b6bb30e5e6bbadc583a9: Status 404 returned error can't find the container with id f718395397b0f754809433a4666b2df69aa89da6e3a6b6bb30e5e6bbadc583a9 Sep 07 13:12:50.348121 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:50.348083 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" event={"ID":"8e135966-53cb-4eca-917d-efcba7c2595f","Type":"ContainerStarted","Data":"76e91b7ec74def4ced47acd5f6ddddcd2448f8f1854dba1dfc2ec5917d4bdc3e"} Sep 07 13:12:50.348316 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:50.348144 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" event={"ID":"8e135966-53cb-4eca-917d-efcba7c2595f","Type":"ContainerStarted","Data":"f718395397b0f754809433a4666b2df69aa89da6e3a6b6bb30e5e6bbadc583a9"} Sep 07 13:12:50.348316 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:50.348174 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" Sep 07 13:12:50.367417 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:12:50.367369 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" podStartSLOduration=1.36735577 podStartE2EDuration="1.36735577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:12:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:12:50.365035974 +0000 UTC m=+1204.395785762" watchObservedRunningTime="2026-09-07 13:12:50.36735577 +0000 UTC m=+1204.398105540" Sep 07 13:13:01.355636 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:01.355605 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" Sep 07 13:13:05.440370 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:05.440339 2576 generic.go:356] "Generic (PLEG): container finished" podID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerID="aa5c4ed0f10dc1cd75db15114b8fb70398ceff0d5fe6a6e38b985fd33adb4735" exitCode=0 Sep 07 13:13:05.440738 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:05.440412 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" event={"ID":"3c45411e-d304-461f-aef1-f6ac915b9a8f","Type":"ContainerDied","Data":"aa5c4ed0f10dc1cd75db15114b8fb70398ceff0d5fe6a6e38b985fd33adb4735"} Sep 07 13:13:05.440837 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:05.440821 2576 scope.go:117] "RemoveContainer" containerID="aa5c4ed0f10dc1cd75db15114b8fb70398ceff0d5fe6a6e38b985fd33adb4735" Sep 07 13:13:05.443832 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:05.442387 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:13:05.443832 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:05.442551 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:13:05.443832 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:05.442646 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:13:06.447747 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:06.447715 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" event={"ID":"3c45411e-d304-461f-aef1-f6ac915b9a8f","Type":"ContainerStarted","Data":"afa412edda94268678da28feb7480b8d682e9d81509e592ff34d229893b48833"} Sep 07 13:13:07.295729 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:07.295679 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:13:07.296466 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:07.296432 2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Sep 07 13:13:07.296615 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:07.296492 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" podUID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Sep 07 13:13:08.151210 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:08.151169 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:13:08.294815 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:08.294776 2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Sep 07 13:13:08.295009 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:08.294838 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" podUID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Sep 07 13:13:09.295357 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:09.295321 2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Sep 07 13:13:09.295748 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:09.295376 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" podUID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Sep 07 13:13:10.295521 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:10.295486 2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Sep 07 13:13:10.295920 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:10.295561 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" podUID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Sep 07 13:13:11.295359 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:11.295320 2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Sep 07 13:13:11.295542 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:11.295375 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" podUID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Sep 07 13:13:12.303179 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:12.303133 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:13:12.310238 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:12.310205 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:13:12.310511 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:12.310488 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:13:12.312095 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:12.312067 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:13:24.953933 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:24.953898 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:13:26.699391 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:26.699347 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:13:28.819608 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:28.819571 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:13:28.919384 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:28.919288 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:13:29.965590 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:29.965554 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:13:30.049153 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.045536 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:13:30.526909 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.526877 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:13:30.635068 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.635033 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2"] Sep 07 13:13:30.637047 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.637006 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:13:30.638880 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.638865 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" Sep 07 13:13:30.640607 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.640587 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-0561b5ad\"" Sep 07 13:13:30.640702 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.640612 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-0561b5ad-dockercfg-l6ms6\"" Sep 07 13:13:30.649282 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.649258 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2"] Sep 07 13:13:30.666892 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.666861 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm"] Sep 07 13:13:30.670873 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.670854 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" Sep 07 13:13:30.680634 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.680607 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm"] Sep 07 13:13:30.712187 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.712160 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e40fce2-8d78-4f12-9462-6ef02e6e28bc-plugins-config-volume\") pod \"payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm\" (UID: \"2e40fce2-8d78-4f12-9462-6ef02e6e28bc\") " pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" Sep 07 13:13:30.712315 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.712287 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss7gp\" (UniqueName: \"kubernetes.io/projected/2e40fce2-8d78-4f12-9462-6ef02e6e28bc-kube-api-access-ss7gp\") pod \"payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm\" (UID: \"2e40fce2-8d78-4f12-9462-6ef02e6e28bc\") " pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" Sep 07 13:13:30.712379 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.712311 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5465020-1eb0-4f1c-8bbf-97bc3685855c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2\" (UID: \"d5465020-1eb0-4f1c-8bbf-97bc3685855c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" Sep 07 13:13:30.712379 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.712334 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwmqb\" (UniqueName: \"kubernetes.io/projected/d5465020-1eb0-4f1c-8bbf-97bc3685855c-kube-api-access-qwmqb\") pod \"payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2\" (UID: \"d5465020-1eb0-4f1c-8bbf-97bc3685855c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" Sep 07 13:13:30.813130 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.813096 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss7gp\" (UniqueName: \"kubernetes.io/projected/2e40fce2-8d78-4f12-9462-6ef02e6e28bc-kube-api-access-ss7gp\") pod \"payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm\" (UID: \"2e40fce2-8d78-4f12-9462-6ef02e6e28bc\") " pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" Sep 07 13:13:30.813130 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.813134 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5465020-1eb0-4f1c-8bbf-97bc3685855c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2\" (UID: \"d5465020-1eb0-4f1c-8bbf-97bc3685855c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" Sep 07 13:13:30.813401 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.813156 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmqb\" (UniqueName: \"kubernetes.io/projected/d5465020-1eb0-4f1c-8bbf-97bc3685855c-kube-api-access-qwmqb\") pod \"payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2\" (UID: \"d5465020-1eb0-4f1c-8bbf-97bc3685855c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" Sep 07 13:13:30.813401 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.813237 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e40fce2-8d78-4f12-9462-6ef02e6e28bc-plugins-config-volume\") pod \"payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm\" (UID: \"2e40fce2-8d78-4f12-9462-6ef02e6e28bc\") " pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" Sep 07 13:13:30.813905 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.813886 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5465020-1eb0-4f1c-8bbf-97bc3685855c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2\" (UID: \"d5465020-1eb0-4f1c-8bbf-97bc3685855c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" Sep 07 13:13:30.814027 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.814007 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e40fce2-8d78-4f12-9462-6ef02e6e28bc-plugins-config-volume\") pod \"payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm\" (UID: \"2e40fce2-8d78-4f12-9462-6ef02e6e28bc\") " pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" Sep 07 13:13:30.822090 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.822069 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwmqb\" (UniqueName: \"kubernetes.io/projected/d5465020-1eb0-4f1c-8bbf-97bc3685855c-kube-api-access-qwmqb\") pod \"payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2\" (UID: \"d5465020-1eb0-4f1c-8bbf-97bc3685855c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" Sep 07 13:13:30.822327 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.822310 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss7gp\" (UniqueName: \"kubernetes.io/projected/2e40fce2-8d78-4f12-9462-6ef02e6e28bc-kube-api-access-ss7gp\") pod \"payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm\" (UID: \"2e40fce2-8d78-4f12-9462-6ef02e6e28bc\") " pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" Sep 07 13:13:30.950071 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.950033 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" Sep 07 13:13:30.981956 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:30.981925 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" Sep 07 13:13:31.229983 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:31.229921 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2"] Sep 07 13:13:31.232180 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:13:31.232150 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5465020_1eb0_4f1c_8bbf_97bc3685855c.slice/crio-809bc709a6752d41b3f98be809884952282fa80d3d4487b0cff4643f66c71c3c WatchSource:0}: Error finding container 809bc709a6752d41b3f98be809884952282fa80d3d4487b0cff4643f66c71c3c: Status 404 returned error can't find the container with id 809bc709a6752d41b3f98be809884952282fa80d3d4487b0cff4643f66c71c3c Sep 07 13:13:31.267222 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:31.267170 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm"] Sep 07 13:13:31.269225 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:13:31.269202 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e40fce2_8d78_4f12_9462_6ef02e6e28bc.slice/crio-0b52210c794ffdc970e2855f493f990b7d231a3ecc2bf4da3ff6c59880fc1e4c WatchSource:0}: Error finding container 0b52210c794ffdc970e2855f493f990b7d231a3ecc2bf4da3ff6c59880fc1e4c: Status 404 returned error can't find the container with id 0b52210c794ffdc970e2855f493f990b7d231a3ecc2bf4da3ff6c59880fc1e4c Sep 07 13:13:31.584680 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:31.584649 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" event={"ID":"2e40fce2-8d78-4f12-9462-6ef02e6e28bc","Type":"ContainerStarted","Data":"f2063d7c8a6c1ce9790c8c51947d7a35c026fc83ded9a001d360c364d202624d"} Sep 07 13:13:31.584901 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:31.584714 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" event={"ID":"2e40fce2-8d78-4f12-9462-6ef02e6e28bc","Type":"ContainerStarted","Data":"0b52210c794ffdc970e2855f493f990b7d231a3ecc2bf4da3ff6c59880fc1e4c"} Sep 07 13:13:31.584901 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:31.584743 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" Sep 07 13:13:31.586817 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:31.586783 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" event={"ID":"d5465020-1eb0-4f1c-8bbf-97bc3685855c","Type":"ContainerStarted","Data":"bcd2e371728d1554c115516db0ff90dacccf845edd493958ddcb6092e125d81c"} Sep 07 13:13:31.586943 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:31.586849 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" Sep 07 13:13:31.586943 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:31.586875 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" event={"ID":"d5465020-1eb0-4f1c-8bbf-97bc3685855c","Type":"ContainerStarted","Data":"809bc709a6752d41b3f98be809884952282fa80d3d4487b0cff4643f66c71c3c"} Sep 07 13:13:31.608640 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:31.608588 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" podStartSLOduration=1.608574062 podStartE2EDuration="1.608574062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:13:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:13:31.607696448 +0000 UTC m=+1245.638446246" watchObservedRunningTime="2026-09-07 13:13:31.608574062 +0000 UTC m=+1245.639323831" Sep 07 13:13:31.638925 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:31.638881 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" podStartSLOduration=1.63886683 podStartE2EDuration="1.63886683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:13:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:13:31.637127492 +0000 UTC m=+1245.667877272" watchObservedRunningTime="2026-09-07 13:13:31.63886683 +0000 UTC m=+1245.669616601" Sep 07 13:13:42.593567 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:42.593481 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" Sep 07 13:13:42.594109 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:42.593726 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" Sep 07 13:13:53.837413 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:13:53.837336 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:14:02.044413 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:02.044369 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:14:02.138315 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:02.138258 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:14:32.581477 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:32.581431 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:14:43.882073 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:43.882035 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:14:46.573397 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:46.573367 2576 scope.go:117] "RemoveContainer" containerID="3c68a91ab88d063a1dc6409c717321e914927896f2904ec674b3f09b1e1e0d1d" Sep 07 13:14:59.479585 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.479530 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:14:59.514828 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.514796 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm"] Sep 07 13:14:59.515289 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.515255 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" podUID="2e40fce2-8d78-4f12-9462-6ef02e6e28bc" containerName="payload-processing" containerID="cri-o://f2063d7c8a6c1ce9790c8c51947d7a35c026fc83ded9a001d360c364d202624d" gracePeriod=30 Sep 07 13:14:59.529721 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.529691 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2"] Sep 07 13:14:59.529896 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.529875 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" podUID="d5465020-1eb0-4f1c-8bbf-97bc3685855c" containerName="payload-pre-processing" containerID="cri-o://bcd2e371728d1554c115516db0ff90dacccf845edd493958ddcb6092e125d81c" gracePeriod=30 Sep 07 13:14:59.564159 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.564127 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:14:59.646585 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.646555 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:14:59.741902 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.741683 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:14:59.811912 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.811869 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:14:59.822583 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.822553 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" Sep 07 13:14:59.825427 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.825407 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" Sep 07 13:14:59.866260 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.866231 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5465020-1eb0-4f1c-8bbf-97bc3685855c-plugins-config-volume\") pod \"d5465020-1eb0-4f1c-8bbf-97bc3685855c\" (UID: \"d5465020-1eb0-4f1c-8bbf-97bc3685855c\") " Sep 07 13:14:59.866440 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.866304 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwmqb\" (UniqueName: \"kubernetes.io/projected/d5465020-1eb0-4f1c-8bbf-97bc3685855c-kube-api-access-qwmqb\") pod \"d5465020-1eb0-4f1c-8bbf-97bc3685855c\" (UID: \"d5465020-1eb0-4f1c-8bbf-97bc3685855c\") " Sep 07 13:14:59.866440 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.866393 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e40fce2-8d78-4f12-9462-6ef02e6e28bc-plugins-config-volume\") pod \"2e40fce2-8d78-4f12-9462-6ef02e6e28bc\" (UID: \"2e40fce2-8d78-4f12-9462-6ef02e6e28bc\") " Sep 07 13:14:59.866568 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.866438 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss7gp\" (UniqueName: \"kubernetes.io/projected/2e40fce2-8d78-4f12-9462-6ef02e6e28bc-kube-api-access-ss7gp\") pod \"2e40fce2-8d78-4f12-9462-6ef02e6e28bc\" (UID: \"2e40fce2-8d78-4f12-9462-6ef02e6e28bc\") " Sep 07 13:14:59.866732 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.866704 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5465020-1eb0-4f1c-8bbf-97bc3685855c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d5465020-1eb0-4f1c-8bbf-97bc3685855c" (UID: "d5465020-1eb0-4f1c-8bbf-97bc3685855c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:14:59.866830 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.866774 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e40fce2-8d78-4f12-9462-6ef02e6e28bc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2e40fce2-8d78-4f12-9462-6ef02e6e28bc" (UID: "2e40fce2-8d78-4f12-9462-6ef02e6e28bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:14:59.868628 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.868603 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5465020-1eb0-4f1c-8bbf-97bc3685855c-kube-api-access-qwmqb" (OuterVolumeSpecName: "kube-api-access-qwmqb") pod "d5465020-1eb0-4f1c-8bbf-97bc3685855c" (UID: "d5465020-1eb0-4f1c-8bbf-97bc3685855c"). InnerVolumeSpecName "kube-api-access-qwmqb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:14:59.868756 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.868734 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e40fce2-8d78-4f12-9462-6ef02e6e28bc-kube-api-access-ss7gp" (OuterVolumeSpecName: "kube-api-access-ss7gp") pod "2e40fce2-8d78-4f12-9462-6ef02e6e28bc" (UID: "2e40fce2-8d78-4f12-9462-6ef02e6e28bc"). InnerVolumeSpecName "kube-api-access-ss7gp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:14:59.967822 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.967789 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwmqb\" (UniqueName: \"kubernetes.io/projected/d5465020-1eb0-4f1c-8bbf-97bc3685855c-kube-api-access-qwmqb\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:14:59.967822 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.967815 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e40fce2-8d78-4f12-9462-6ef02e6e28bc-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:14:59.967822 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.967824 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss7gp\" (UniqueName: \"kubernetes.io/projected/2e40fce2-8d78-4f12-9462-6ef02e6e28bc-kube-api-access-ss7gp\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:14:59.968096 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:14:59.967833 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5465020-1eb0-4f1c-8bbf-97bc3685855c-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:15:00.070656 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.070625 2576 generic.go:356] "Generic (PLEG): container finished" podID="d5465020-1eb0-4f1c-8bbf-97bc3685855c" containerID="bcd2e371728d1554c115516db0ff90dacccf845edd493958ddcb6092e125d81c" exitCode=0 Sep 07 13:15:00.070830 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.070670 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" Sep 07 13:15:00.070830 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.070671 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" event={"ID":"d5465020-1eb0-4f1c-8bbf-97bc3685855c","Type":"ContainerDied","Data":"bcd2e371728d1554c115516db0ff90dacccf845edd493958ddcb6092e125d81c"} Sep 07 13:15:00.070830 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.070735 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2" event={"ID":"d5465020-1eb0-4f1c-8bbf-97bc3685855c","Type":"ContainerDied","Data":"809bc709a6752d41b3f98be809884952282fa80d3d4487b0cff4643f66c71c3c"} Sep 07 13:15:00.070830 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.070771 2576 scope.go:117] "RemoveContainer" containerID="bcd2e371728d1554c115516db0ff90dacccf845edd493958ddcb6092e125d81c" Sep 07 13:15:00.072035 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.071951 2576 generic.go:356] "Generic (PLEG): container finished" podID="2e40fce2-8d78-4f12-9462-6ef02e6e28bc" containerID="f2063d7c8a6c1ce9790c8c51947d7a35c026fc83ded9a001d360c364d202624d" exitCode=0 Sep 07 13:15:00.072035 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.072009 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" Sep 07 13:15:00.072035 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.072011 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" event={"ID":"2e40fce2-8d78-4f12-9462-6ef02e6e28bc","Type":"ContainerDied","Data":"f2063d7c8a6c1ce9790c8c51947d7a35c026fc83ded9a001d360c364d202624d"} Sep 07 13:15:00.072194 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.072053 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm" event={"ID":"2e40fce2-8d78-4f12-9462-6ef02e6e28bc","Type":"ContainerDied","Data":"0b52210c794ffdc970e2855f493f990b7d231a3ecc2bf4da3ff6c59880fc1e4c"} Sep 07 13:15:00.081234 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.081213 2576 scope.go:117] "RemoveContainer" containerID="bcd2e371728d1554c115516db0ff90dacccf845edd493958ddcb6092e125d81c" Sep 07 13:15:00.081565 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:15:00.081510 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcd2e371728d1554c115516db0ff90dacccf845edd493958ddcb6092e125d81c\": container with ID starting with bcd2e371728d1554c115516db0ff90dacccf845edd493958ddcb6092e125d81c not found: ID does not exist" containerID="bcd2e371728d1554c115516db0ff90dacccf845edd493958ddcb6092e125d81c" Sep 07 13:15:00.081565 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.081541 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd2e371728d1554c115516db0ff90dacccf845edd493958ddcb6092e125d81c"} err="failed to get container status \"bcd2e371728d1554c115516db0ff90dacccf845edd493958ddcb6092e125d81c\": rpc error: code = NotFound desc = could not find container \"bcd2e371728d1554c115516db0ff90dacccf845edd493958ddcb6092e125d81c\": container with ID starting with bcd2e371728d1554c115516db0ff90dacccf845edd493958ddcb6092e125d81c not found: ID does not exist" Sep 07 13:15:00.081565 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.081557 2576 scope.go:117] "RemoveContainer" containerID="f2063d7c8a6c1ce9790c8c51947d7a35c026fc83ded9a001d360c364d202624d" Sep 07 13:15:00.089687 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.089670 2576 scope.go:117] "RemoveContainer" containerID="f2063d7c8a6c1ce9790c8c51947d7a35c026fc83ded9a001d360c364d202624d" Sep 07 13:15:00.089929 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:15:00.089909 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2063d7c8a6c1ce9790c8c51947d7a35c026fc83ded9a001d360c364d202624d\": container with ID starting with f2063d7c8a6c1ce9790c8c51947d7a35c026fc83ded9a001d360c364d202624d not found: ID does not exist" containerID="f2063d7c8a6c1ce9790c8c51947d7a35c026fc83ded9a001d360c364d202624d" Sep 07 13:15:00.090028 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.089939 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2063d7c8a6c1ce9790c8c51947d7a35c026fc83ded9a001d360c364d202624d"} err="failed to get container status \"f2063d7c8a6c1ce9790c8c51947d7a35c026fc83ded9a001d360c364d202624d\": rpc error: code = NotFound desc = could not find container \"f2063d7c8a6c1ce9790c8c51947d7a35c026fc83ded9a001d360c364d202624d\": container with ID starting with f2063d7c8a6c1ce9790c8c51947d7a35c026fc83ded9a001d360c364d202624d not found: ID does not exist" Sep 07 13:15:00.097238 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.097215 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm"] Sep 07 13:15:00.102199 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.102179 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0561b5ad-66f4b746cc-hxbdm"] Sep 07 13:15:00.115889 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.115854 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2"] Sep 07 13:15:00.117673 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.117655 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0561b5ad-84bd568458-5tpd2"] Sep 07 13:15:00.149399 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.149362 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29813115-7l9vq"] Sep 07 13:15:00.149791 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.149777 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e40fce2-8d78-4f12-9462-6ef02e6e28bc" containerName="payload-processing" Sep 07 13:15:00.149856 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.149793 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e40fce2-8d78-4f12-9462-6ef02e6e28bc" containerName="payload-processing" Sep 07 13:15:00.149856 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.149815 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5465020-1eb0-4f1c-8bbf-97bc3685855c" containerName="payload-pre-processing" Sep 07 13:15:00.149856 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.149823 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5465020-1eb0-4f1c-8bbf-97bc3685855c" containerName="payload-pre-processing" Sep 07 13:15:00.149951 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.149903 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5465020-1eb0-4f1c-8bbf-97bc3685855c" containerName="payload-pre-processing" Sep 07 13:15:00.149951 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.149914 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e40fce2-8d78-4f12-9462-6ef02e6e28bc" containerName="payload-processing" Sep 07 13:15:00.154510 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.154490 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr"] Sep 07 13:15:00.154624 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.154607 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813115-7l9vq" Sep 07 13:15:00.157331 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.157311 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-flngs\"" Sep 07 13:15:00.157740 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.157722 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr" Sep 07 13:15:00.158852 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.158833 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl"] Sep 07 13:15:00.162193 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.162175 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl" Sep 07 13:15:00.167758 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.167682 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813115-7l9vq" Sep 07 13:15:00.172533 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.169190 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29813115-7l9vq"] Sep 07 13:15:00.181924 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.177398 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr"] Sep 07 13:15:00.188874 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.183886 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl"] Sep 07 13:15:00.190985 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.190947 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr" Sep 07 13:15:00.199119 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.199097 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl" Sep 07 13:15:00.467980 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.467943 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29813115-7l9vq"] Sep 07 13:15:00.471002 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:15:00.470927 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71a0d607_780b_474c_9874_cabc806c3f3a.slice/crio-db53037eb10f742c7b4ff36883c94fab0bd584a7f60f45e680dff5e6368644d2 WatchSource:0}: Error finding container db53037eb10f742c7b4ff36883c94fab0bd584a7f60f45e680dff5e6368644d2: Status 404 returned error can't find the container with id db53037eb10f742c7b4ff36883c94fab0bd584a7f60f45e680dff5e6368644d2 Sep 07 13:15:00.477846 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:15:00.477820 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c61ccac_366d_4868_8e11_35d32d239bb0.slice/crio-d9d5ba2632b4d31686e80b54765f14cecd24658ff337bc7d3f1a5100c49399ca WatchSource:0}: Error finding container d9d5ba2632b4d31686e80b54765f14cecd24658ff337bc7d3f1a5100c49399ca: Status 404 returned error can't find the container with id d9d5ba2632b4d31686e80b54765f14cecd24658ff337bc7d3f1a5100c49399ca Sep 07 13:15:00.483441 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.483415 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e40fce2-8d78-4f12-9462-6ef02e6e28bc" path="/var/lib/kubelet/pods/2e40fce2-8d78-4f12-9462-6ef02e6e28bc/volumes" Sep 07 13:15:00.483901 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.483834 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5465020-1eb0-4f1c-8bbf-97bc3685855c" path="/var/lib/kubelet/pods/d5465020-1eb0-4f1c-8bbf-97bc3685855c/volumes" Sep 07 13:15:00.484212 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.484196 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr"] Sep 07 13:15:00.491142 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:00.491095 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl"] Sep 07 13:15:00.494475 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:15:00.494450 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45207102_69f0_4c12_9865_b4e470feaf8b.slice/crio-e5bba0cba632c9e6c81f924532b26c39a7b0e08a886acc978a80f304a35f8a68 WatchSource:0}: Error finding container e5bba0cba632c9e6c81f924532b26c39a7b0e08a886acc978a80f304a35f8a68: Status 404 returned error can't find the container with id e5bba0cba632c9e6c81f924532b26c39a7b0e08a886acc978a80f304a35f8a68 Sep 07 13:15:01.080609 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:01.080513 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr" event={"ID":"0c61ccac-366d-4868-8e11-35d32d239bb0","Type":"ContainerStarted","Data":"6d6da8255e12bcb56e3df17597d62e1113646fe3f93a26cbdd318fabcc02efd1"} Sep 07 13:15:01.080609 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:01.080568 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr" event={"ID":"0c61ccac-366d-4868-8e11-35d32d239bb0","Type":"ContainerStarted","Data":"d9d5ba2632b4d31686e80b54765f14cecd24658ff337bc7d3f1a5100c49399ca"} Sep 07 13:15:01.082098 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:01.082066 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl" event={"ID":"45207102-69f0-4c12-9865-b4e470feaf8b","Type":"ContainerStarted","Data":"69619efb94c4c166c96102430fe90c0fd1a373d4c0c71feefb0f8de823d7f843"} Sep 07 13:15:01.082209 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:01.082122 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl" event={"ID":"45207102-69f0-4c12-9865-b4e470feaf8b","Type":"ContainerStarted","Data":"e5bba0cba632c9e6c81f924532b26c39a7b0e08a886acc978a80f304a35f8a68"} Sep 07 13:15:01.083520 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:01.083490 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813115-7l9vq" event={"ID":"71a0d607-780b-474c-9874-cabc806c3f3a","Type":"ContainerStarted","Data":"c75b95c598283207f88d3fac83dd288430f9e603d9223050a2acc998ce1bdf58"} Sep 07 13:15:01.083649 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:01.083535 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813115-7l9vq" event={"ID":"71a0d607-780b-474c-9874-cabc806c3f3a","Type":"ContainerStarted","Data":"db53037eb10f742c7b4ff36883c94fab0bd584a7f60f45e680dff5e6368644d2"} Sep 07 13:15:01.097694 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:01.097653 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr" podStartSLOduration=1.097641381 podStartE2EDuration="1.097641381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:15:01.095571667 +0000 UTC m=+1335.126321451" watchObservedRunningTime="2026-09-07 13:15:01.097641381 +0000 UTC m=+1335.128391146" Sep 07 13:15:01.113268 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:01.113225 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813115-7l9vq" podStartSLOduration=1.11321183 podStartE2EDuration="1.11321183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:15:01.111712348 +0000 UTC m=+1335.142462118" watchObservedRunningTime="2026-09-07 13:15:01.11321183 +0000 UTC m=+1335.143961601" Sep 07 13:15:01.137320 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:01.137278 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl" podStartSLOduration=1.137265025 podStartE2EDuration="1.137265025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:15:01.135013424 +0000 UTC m=+1335.165763211" watchObservedRunningTime="2026-09-07 13:15:01.137265025 +0000 UTC m=+1335.168014798" Sep 07 13:15:02.091392 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:02.091362 2576 generic.go:356] "Generic (PLEG): container finished" podID="0c61ccac-366d-4868-8e11-35d32d239bb0" containerID="6d6da8255e12bcb56e3df17597d62e1113646fe3f93a26cbdd318fabcc02efd1" exitCode=0 Sep 07 13:15:02.091791 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:02.091440 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr" event={"ID":"0c61ccac-366d-4868-8e11-35d32d239bb0","Type":"ContainerDied","Data":"6d6da8255e12bcb56e3df17597d62e1113646fe3f93a26cbdd318fabcc02efd1"} Sep 07 13:15:02.092961 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:02.092939 2576 generic.go:356] "Generic (PLEG): container finished" podID="45207102-69f0-4c12-9865-b4e470feaf8b" containerID="69619efb94c4c166c96102430fe90c0fd1a373d4c0c71feefb0f8de823d7f843" exitCode=0 Sep 07 13:15:02.093094 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:02.093001 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl" event={"ID":"45207102-69f0-4c12-9865-b4e470feaf8b","Type":"ContainerDied","Data":"69619efb94c4c166c96102430fe90c0fd1a373d4c0c71feefb0f8de823d7f843"} Sep 07 13:15:02.094592 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:02.094572 2576 generic.go:356] "Generic (PLEG): container finished" podID="71a0d607-780b-474c-9874-cabc806c3f3a" containerID="c75b95c598283207f88d3fac83dd288430f9e603d9223050a2acc998ce1bdf58" exitCode=0 Sep 07 13:15:02.094683 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:02.094655 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813115-7l9vq" event={"ID":"71a0d607-780b-474c-9874-cabc806c3f3a","Type":"ContainerDied","Data":"c75b95c598283207f88d3fac83dd288430f9e603d9223050a2acc998ce1bdf58"} Sep 07 13:15:03.245771 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:03.245738 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:15:03.255762 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:03.255740 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr" Sep 07 13:15:03.294682 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:03.294647 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813115-7l9vq" Sep 07 13:15:03.297581 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:03.297559 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl" Sep 07 13:15:04.107626 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:04.107592 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl" Sep 07 13:15:04.107828 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:04.107601 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl" event={"ID":"45207102-69f0-4c12-9865-b4e470feaf8b","Type":"ContainerDied","Data":"e5bba0cba632c9e6c81f924532b26c39a7b0e08a886acc978a80f304a35f8a68"} Sep 07 13:15:04.107828 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:04.107743 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5bba0cba632c9e6c81f924532b26c39a7b0e08a886acc978a80f304a35f8a68" Sep 07 13:15:04.108990 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:04.108957 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813115-7l9vq" Sep 07 13:15:04.109094 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:04.108994 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813115-7l9vq" event={"ID":"71a0d607-780b-474c-9874-cabc806c3f3a","Type":"ContainerDied","Data":"db53037eb10f742c7b4ff36883c94fab0bd584a7f60f45e680dff5e6368644d2"} Sep 07 13:15:04.109094 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:04.109028 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db53037eb10f742c7b4ff36883c94fab0bd584a7f60f45e680dff5e6368644d2" Sep 07 13:15:04.110414 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:04.110391 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr" event={"ID":"0c61ccac-366d-4868-8e11-35d32d239bb0","Type":"ContainerDied","Data":"d9d5ba2632b4d31686e80b54765f14cecd24658ff337bc7d3f1a5100c49399ca"} Sep 07 13:15:04.110414 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:04.110409 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr" Sep 07 13:15:04.110585 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:04.110429 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9d5ba2632b4d31686e80b54765f14cecd24658ff337bc7d3f1a5100c49399ca" Sep 07 13:15:08.304315 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:08.304230 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:15:21.454153 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:21.454118 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:15:22.824113 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:22.824078 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:15:27.211194 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:27.211159 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:15:47.575582 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:47.575544 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:15:47.759591 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:47.759551 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:15:52.366348 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:52.366314 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:15:54.645632 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:15:54.645599 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:00.960837 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:00.960804 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:01.065753 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:01.065715 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:01.279342 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:01.279310 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:02.668318 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:02.668280 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:02.763228 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:02.763187 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:02.908919 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:02.908887 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:03.731018 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:03.730988 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:03.823743 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:03.823709 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:05.273571 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:05.273535 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:11.952165 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:11.952133 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:12.053965 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:12.053923 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:12.679956 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:12.679914 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:19.388515 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:19.388482 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:19.481629 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:19.481596 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:30.062643 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:30.062607 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:30.327929 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:30.327821 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:38.733866 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:38.733777 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:39.737266 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:39.737220 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:41.076493 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:41.076446 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:44.798450 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:44.798417 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:46.612375 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:46.612347 2576 scope.go:117] "RemoveContainer" containerID="a56f81841c99322b182071bd66f10920cb22ec133dac003fbe28852aa538a509" Sep 07 13:16:46.620888 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:46.620869 2576 scope.go:117] "RemoveContainer" containerID="d771da394f1aae0384521434906fec5c451ba215a3f1ab276eeb4fef556d6f87" Sep 07 13:16:50.056949 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:50.056917 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:51.370204 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:51.370171 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:57.967243 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:57.967200 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:58.087660 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:58.087624 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:58.348911 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:58.348870 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:58.739057 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:58.738908 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:16:58.948608 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:16:58.948563 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:02.142420 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:02.142382 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:02.234468 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:02.234438 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:03.453613 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:03.453573 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:03.531085 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:03.531054 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:03.829159 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:03.829119 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:04.067996 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.067931 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:04.187692 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.187609 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl"] Sep 07 13:17:04.188067 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.188052 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45207102-69f0-4c12-9865-b4e470feaf8b" containerName="cleanup" Sep 07 13:17:04.188130 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.188069 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="45207102-69f0-4c12-9865-b4e470feaf8b" containerName="cleanup" Sep 07 13:17:04.188130 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.188081 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71a0d607-780b-474c-9874-cabc806c3f3a" containerName="cleanup" Sep 07 13:17:04.188130 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.188088 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="71a0d607-780b-474c-9874-cabc806c3f3a" containerName="cleanup" Sep 07 13:17:04.188130 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.188105 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c61ccac-366d-4868-8e11-35d32d239bb0" containerName="cleanup" Sep 07 13:17:04.188130 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.188113 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c61ccac-366d-4868-8e11-35d32d239bb0" containerName="cleanup" Sep 07 13:17:04.188298 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.188203 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="45207102-69f0-4c12-9865-b4e470feaf8b" containerName="cleanup" Sep 07 13:17:04.188298 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.188212 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="71a0d607-780b-474c-9874-cabc806c3f3a" containerName="cleanup" Sep 07 13:17:04.188298 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.188219 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c61ccac-366d-4868-8e11-35d32d239bb0" containerName="cleanup" Sep 07 13:17:04.191885 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.191860 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:04.192610 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.192596 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" Sep 07 13:17:04.194876 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.194856 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ddb9ca\"" Sep 07 13:17:04.211867 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.211846 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl"] Sep 07 13:17:04.244931 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.244895 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp"] Sep 07 13:17:04.245111 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.244928 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b05aa5e9-207a-4f43-9030-05b6b9f104ca-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl\" (UID: \"b05aa5e9-207a-4f43-9030-05b6b9f104ca\") " pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" Sep 07 13:17:04.245111 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.245008 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmj98\" (UniqueName: \"kubernetes.io/projected/b05aa5e9-207a-4f43-9030-05b6b9f104ca-kube-api-access-mmj98\") pod \"payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl\" (UID: \"b05aa5e9-207a-4f43-9030-05b6b9f104ca\") " pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" Sep 07 13:17:04.248366 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.248349 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" Sep 07 13:17:04.250772 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.250756 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ddb9ca-dockercfg-xqnmz\"" Sep 07 13:17:04.256506 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.256485 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp"] Sep 07 13:17:04.345903 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.345861 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f21d3c6-2d63-4a11-943d-79d446f3ba58-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp\" (UID: \"5f21d3c6-2d63-4a11-943d-79d446f3ba58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" Sep 07 13:17:04.346142 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.345910 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmj98\" (UniqueName: \"kubernetes.io/projected/b05aa5e9-207a-4f43-9030-05b6b9f104ca-kube-api-access-mmj98\") pod \"payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl\" (UID: \"b05aa5e9-207a-4f43-9030-05b6b9f104ca\") " pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" Sep 07 13:17:04.346142 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.345934 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76kn8\" (UniqueName: \"kubernetes.io/projected/5f21d3c6-2d63-4a11-943d-79d446f3ba58-kube-api-access-76kn8\") pod \"payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp\" (UID: \"5f21d3c6-2d63-4a11-943d-79d446f3ba58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" Sep 07 13:17:04.346142 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.346069 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b05aa5e9-207a-4f43-9030-05b6b9f104ca-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl\" (UID: \"b05aa5e9-207a-4f43-9030-05b6b9f104ca\") " pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" Sep 07 13:17:04.346791 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.346766 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b05aa5e9-207a-4f43-9030-05b6b9f104ca-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl\" (UID: \"b05aa5e9-207a-4f43-9030-05b6b9f104ca\") " pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" Sep 07 13:17:04.357656 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.357635 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmj98\" (UniqueName: \"kubernetes.io/projected/b05aa5e9-207a-4f43-9030-05b6b9f104ca-kube-api-access-mmj98\") pod \"payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl\" (UID: \"b05aa5e9-207a-4f43-9030-05b6b9f104ca\") " pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" Sep 07 13:17:04.446648 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.446549 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f21d3c6-2d63-4a11-943d-79d446f3ba58-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp\" (UID: \"5f21d3c6-2d63-4a11-943d-79d446f3ba58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" Sep 07 13:17:04.446648 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.446593 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76kn8\" (UniqueName: \"kubernetes.io/projected/5f21d3c6-2d63-4a11-943d-79d446f3ba58-kube-api-access-76kn8\") pod \"payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp\" (UID: \"5f21d3c6-2d63-4a11-943d-79d446f3ba58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" Sep 07 13:17:04.447326 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.447307 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f21d3c6-2d63-4a11-943d-79d446f3ba58-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp\" (UID: \"5f21d3c6-2d63-4a11-943d-79d446f3ba58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" Sep 07 13:17:04.458238 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.458211 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76kn8\" (UniqueName: \"kubernetes.io/projected/5f21d3c6-2d63-4a11-943d-79d446f3ba58-kube-api-access-76kn8\") pod \"payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp\" (UID: \"5f21d3c6-2d63-4a11-943d-79d446f3ba58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" Sep 07 13:17:04.502721 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.502674 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" Sep 07 13:17:04.559579 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.559543 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" Sep 07 13:17:04.786025 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.785996 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl"] Sep 07 13:17:04.788674 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:17:04.788639 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb05aa5e9_207a_4f43_9030_05b6b9f104ca.slice/crio-9b16431d3468b8ad373c0185d92832b522ab99d7cd65a5c3cb748cdc6c73bf47 WatchSource:0}: Error finding container 9b16431d3468b8ad373c0185d92832b522ab99d7cd65a5c3cb748cdc6c73bf47: Status 404 returned error can't find the container with id 9b16431d3468b8ad373c0185d92832b522ab99d7cd65a5c3cb748cdc6c73bf47 Sep 07 13:17:04.853704 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:04.853677 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp"] Sep 07 13:17:04.856833 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:17:04.856804 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f21d3c6_2d63_4a11_943d_79d446f3ba58.slice/crio-68a0f1ecf9ff477b2185443341e07854e95e26f92d295db200ef79a4cb1bf869 WatchSource:0}: Error finding container 68a0f1ecf9ff477b2185443341e07854e95e26f92d295db200ef79a4cb1bf869: Status 404 returned error can't find the container with id 68a0f1ecf9ff477b2185443341e07854e95e26f92d295db200ef79a4cb1bf869 Sep 07 13:17:05.754117 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:05.754075 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" event={"ID":"b05aa5e9-207a-4f43-9030-05b6b9f104ca","Type":"ContainerStarted","Data":"4e953c1056c57cce163381caffe6bbb0a72785340d696029eba5c172875d0f67"} Sep 07 13:17:05.754619 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:05.754160 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" Sep 07 13:17:05.754619 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:05.754186 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" event={"ID":"b05aa5e9-207a-4f43-9030-05b6b9f104ca","Type":"ContainerStarted","Data":"9b16431d3468b8ad373c0185d92832b522ab99d7cd65a5c3cb748cdc6c73bf47"} Sep 07 13:17:05.755691 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:05.755664 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" event={"ID":"5f21d3c6-2d63-4a11-943d-79d446f3ba58","Type":"ContainerStarted","Data":"9a71d2a5eb538ce6e62d71e43d1f2f2f2ba4d2e48e9402fb929722e5256e70e5"} Sep 07 13:17:05.755853 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:05.755717 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" event={"ID":"5f21d3c6-2d63-4a11-943d-79d446f3ba58","Type":"ContainerStarted","Data":"68a0f1ecf9ff477b2185443341e07854e95e26f92d295db200ef79a4cb1bf869"} Sep 07 13:17:05.755853 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:05.755748 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" Sep 07 13:17:05.773153 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:05.773093 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" podStartSLOduration=1.773069896 podStartE2EDuration="1.773069896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:17:05.772065807 +0000 UTC m=+1459.802815599" watchObservedRunningTime="2026-09-07 13:17:05.773069896 +0000 UTC m=+1459.803819668" Sep 07 13:17:05.795338 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:05.795294 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" podStartSLOduration=1.795279938 podStartE2EDuration="1.795279938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:17:05.793443005 +0000 UTC m=+1459.824192802" watchObservedRunningTime="2026-09-07 13:17:05.795279938 +0000 UTC m=+1459.826029710" Sep 07 13:17:11.344085 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:11.344049 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:16.764515 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:16.764478 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" Sep 07 13:17:16.764888 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:16.764664 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" Sep 07 13:17:24.685637 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:24.685600 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:24.976311 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:24.976222 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:27.252596 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:27.252563 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:27.874761 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:27.874723 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:28.127082 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:28.126986 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:28.392127 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:28.392034 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:31.700515 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:31.700476 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:31.800352 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:31.800316 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:32.589026 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:32.588996 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:32.690862 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:32.690811 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:33.044026 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.043997 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:33.227328 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.227292 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:33.323739 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.323656 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:33.353863 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.353834 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w"] Sep 07 13:17:33.357452 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.357434 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" Sep 07 13:17:33.359067 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.359045 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-0d2b4b\"" Sep 07 13:17:33.359221 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.359204 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-0d2b4b-dockercfg-cfsrv\"" Sep 07 13:17:33.368018 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.367996 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w"] Sep 07 13:17:33.528622 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.528579 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcr8c\" (UniqueName: \"kubernetes.io/projected/547e1e0a-513d-4775-8e7a-dd5a545b3b8a-kube-api-access-rcr8c\") pod \"payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w\" (UID: \"547e1e0a-513d-4775-8e7a-dd5a545b3b8a\") " pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" Sep 07 13:17:33.528801 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.528645 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547e1e0a-513d-4775-8e7a-dd5a545b3b8a-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w\" (UID: \"547e1e0a-513d-4775-8e7a-dd5a545b3b8a\") " pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" Sep 07 13:17:33.629740 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.629653 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcr8c\" (UniqueName: \"kubernetes.io/projected/547e1e0a-513d-4775-8e7a-dd5a545b3b8a-kube-api-access-rcr8c\") pod \"payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w\" (UID: \"547e1e0a-513d-4775-8e7a-dd5a545b3b8a\") " pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" Sep 07 13:17:33.629740 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.629702 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547e1e0a-513d-4775-8e7a-dd5a545b3b8a-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w\" (UID: \"547e1e0a-513d-4775-8e7a-dd5a545b3b8a\") " pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" Sep 07 13:17:33.630435 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.630415 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547e1e0a-513d-4775-8e7a-dd5a545b3b8a-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w\" (UID: \"547e1e0a-513d-4775-8e7a-dd5a545b3b8a\") " pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" Sep 07 13:17:33.639355 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.639332 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcr8c\" (UniqueName: \"kubernetes.io/projected/547e1e0a-513d-4775-8e7a-dd5a545b3b8a-kube-api-access-rcr8c\") pod \"payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w\" (UID: \"547e1e0a-513d-4775-8e7a-dd5a545b3b8a\") " pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" Sep 07 13:17:33.669296 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.669269 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" Sep 07 13:17:33.946488 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:33.946462 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w"] Sep 07 13:17:33.948416 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:17:33.948375 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod547e1e0a_513d_4775_8e7a_dd5a545b3b8a.slice/crio-b5aa4c47ff7f1a4dfeb4b5558bdb0b28bf0df4be673889747b28f8d49fe2eae5 WatchSource:0}: Error finding container b5aa4c47ff7f1a4dfeb4b5558bdb0b28bf0df4be673889747b28f8d49fe2eae5: Status 404 returned error can't find the container with id b5aa4c47ff7f1a4dfeb4b5558bdb0b28bf0df4be673889747b28f8d49fe2eae5 Sep 07 13:17:34.917905 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:34.917870 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" event={"ID":"547e1e0a-513d-4775-8e7a-dd5a545b3b8a","Type":"ContainerStarted","Data":"c84b829a63ab7f35f90d8a822872ffc8655d7c89896005d53f02d50cfc389f1a"} Sep 07 13:17:34.918390 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:34.917931 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" event={"ID":"547e1e0a-513d-4775-8e7a-dd5a545b3b8a","Type":"ContainerStarted","Data":"b5aa4c47ff7f1a4dfeb4b5558bdb0b28bf0df4be673889747b28f8d49fe2eae5"} Sep 07 13:17:34.918390 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:34.918099 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" Sep 07 13:17:34.945812 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:34.945764 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" podStartSLOduration=1.9457480010000001 podStartE2EDuration="1.945748001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:17:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:17:34.944316161 +0000 UTC m=+1488.975065935" watchObservedRunningTime="2026-09-07 13:17:34.945748001 +0000 UTC m=+1488.976497772" Sep 07 13:17:41.115844 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:41.115801 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:45.926128 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:45.926099 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" Sep 07 13:17:46.516755 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:46.516725 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 13:17:46.519075 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:46.519052 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 13:17:46.643479 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:46.643448 2576 scope.go:117] "RemoveContainer" containerID="2f8df9a0cfd0ab8bcb4b13409465e60d5dc764aa7d1425f9e3484b1a7e8bcff2" Sep 07 13:17:53.842499 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:53.842462 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:56.441770 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:56.441729 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:59.762096 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:59.762065 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj"] Sep 07 13:17:59.766957 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:59.766938 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj" Sep 07 13:17:59.768823 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:59.768800 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 13:17:59.768945 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:59.768803 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-flngs\"" Sep 07 13:17:59.776548 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:59.776521 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj"] Sep 07 13:17:59.879805 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:59.879769 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f13e66-dd38-4e34-a068-c659a3e7d223-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj\" (UID: \"53f13e66-dd38-4e34-a068-c659a3e7d223\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj" Sep 07 13:17:59.913265 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:59.913237 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:17:59.980519 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:59.980479 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f13e66-dd38-4e34-a068-c659a3e7d223-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj\" (UID: \"53f13e66-dd38-4e34-a068-c659a3e7d223\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj" Sep 07 13:17:59.981145 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:17:59.981125 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f13e66-dd38-4e34-a068-c659a3e7d223-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj\" (UID: \"53f13e66-dd38-4e34-a068-c659a3e7d223\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj" Sep 07 13:18:00.079016 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:00.078965 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj" Sep 07 13:18:00.355027 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:00.355001 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj"] Sep 07 13:18:00.357417 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:18:00.357382 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53f13e66_dd38_4e34_a068_c659a3e7d223.slice/crio-3f4c89fd64dd087d231026c52be4ab3575bb3cf6e6e4d2046f6bdd31b8806ca2 WatchSource:0}: Error finding container 3f4c89fd64dd087d231026c52be4ab3575bb3cf6e6e4d2046f6bdd31b8806ca2: Status 404 returned error can't find the container with id 3f4c89fd64dd087d231026c52be4ab3575bb3cf6e6e4d2046f6bdd31b8806ca2 Sep 07 13:18:01.060418 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:01.060382 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj" event={"ID":"53f13e66-dd38-4e34-a068-c659a3e7d223","Type":"ContainerStarted","Data":"bb31d5e699fe3efeff58b781b6375245e6ec22c153d135c16075bf995451d937"} Sep 07 13:18:01.060806 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:01.060440 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj" event={"ID":"53f13e66-dd38-4e34-a068-c659a3e7d223","Type":"ContainerStarted","Data":"3f4c89fd64dd087d231026c52be4ab3575bb3cf6e6e4d2046f6bdd31b8806ca2"} Sep 07 13:18:01.075415 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:01.075201 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj" podStartSLOduration=2.075179474 podStartE2EDuration="2.075179474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:17:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:18:01.074844092 +0000 UTC m=+1515.105593874" watchObservedRunningTime="2026-09-07 13:18:01.075179474 +0000 UTC m=+1515.105929249" Sep 07 13:18:02.067282 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:02.067242 2576 generic.go:356] "Generic (PLEG): container finished" podID="53f13e66-dd38-4e34-a068-c659a3e7d223" containerID="bb31d5e699fe3efeff58b781b6375245e6ec22c153d135c16075bf995451d937" exitCode=0 Sep 07 13:18:02.067666 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:02.067329 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj" event={"ID":"53f13e66-dd38-4e34-a068-c659a3e7d223","Type":"ContainerDied","Data":"bb31d5e699fe3efeff58b781b6375245e6ec22c153d135c16075bf995451d937"} Sep 07 13:18:03.217899 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:03.217862 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj" Sep 07 13:18:03.315059 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:03.315017 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f13e66-dd38-4e34-a068-c659a3e7d223-maas-api-service-ca\") pod \"53f13e66-dd38-4e34-a068-c659a3e7d223\" (UID: \"53f13e66-dd38-4e34-a068-c659a3e7d223\") " Sep 07 13:18:03.315400 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:03.315377 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53f13e66-dd38-4e34-a068-c659a3e7d223-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "53f13e66-dd38-4e34-a068-c659a3e7d223" (UID: "53f13e66-dd38-4e34-a068-c659a3e7d223"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:18:03.416809 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:03.416716 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53f13e66-dd38-4e34-a068-c659a3e7d223-maas-api-service-ca\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:18:04.079542 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:04.079511 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj" Sep 07 13:18:04.079725 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:04.079515 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj" event={"ID":"53f13e66-dd38-4e34-a068-c659a3e7d223","Type":"ContainerDied","Data":"3f4c89fd64dd087d231026c52be4ab3575bb3cf6e6e4d2046f6bdd31b8806ca2"} Sep 07 13:18:04.079725 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:04.079631 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f4c89fd64dd087d231026c52be4ab3575bb3cf6e6e4d2046f6bdd31b8806ca2" Sep 07 13:18:07.066808 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:07.066771 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:07.205805 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:07.205770 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:10.515353 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:10.515260 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:16.237403 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.237360 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:16.260679 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.260642 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl"] Sep 07 13:18:16.260914 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.260866 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" podUID="b05aa5e9-207a-4f43-9030-05b6b9f104ca" containerName="payload-processing" containerID="cri-o://4e953c1056c57cce163381caffe6bbb0a72785340d696029eba5c172875d0f67" gracePeriod=30 Sep 07 13:18:16.268099 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.267732 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp"] Sep 07 13:18:16.268099 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.267988 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" podUID="5f21d3c6-2d63-4a11-943d-79d446f3ba58" containerName="payload-pre-processing" containerID="cri-o://9a71d2a5eb538ce6e62d71e43d1f2f2f2ba4d2e48e9402fb929722e5256e70e5" gracePeriod=30 Sep 07 13:18:16.322330 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.322296 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:16.422070 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.422028 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:16.456955 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.456916 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj"] Sep 07 13:18:16.459144 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.459123 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb9ca-22jtj"] Sep 07 13:18:16.485556 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.485528 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53f13e66-dd38-4e34-a068-c659a3e7d223" path="/var/lib/kubelet/pods/53f13e66-dd38-4e34-a068-c659a3e7d223/volumes" Sep 07 13:18:16.523599 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.523567 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:16.545040 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.545016 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" Sep 07 13:18:16.550129 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.550108 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" Sep 07 13:18:16.645997 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.645937 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b05aa5e9-207a-4f43-9030-05b6b9f104ca-plugins-config-volume\") pod \"b05aa5e9-207a-4f43-9030-05b6b9f104ca\" (UID: \"b05aa5e9-207a-4f43-9030-05b6b9f104ca\") " Sep 07 13:18:16.646206 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.646004 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmj98\" (UniqueName: \"kubernetes.io/projected/b05aa5e9-207a-4f43-9030-05b6b9f104ca-kube-api-access-mmj98\") pod \"b05aa5e9-207a-4f43-9030-05b6b9f104ca\" (UID: \"b05aa5e9-207a-4f43-9030-05b6b9f104ca\") " Sep 07 13:18:16.646206 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.646076 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f21d3c6-2d63-4a11-943d-79d446f3ba58-plugins-config-volume\") pod \"5f21d3c6-2d63-4a11-943d-79d446f3ba58\" (UID: \"5f21d3c6-2d63-4a11-943d-79d446f3ba58\") " Sep 07 13:18:16.646206 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.646107 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76kn8\" (UniqueName: \"kubernetes.io/projected/5f21d3c6-2d63-4a11-943d-79d446f3ba58-kube-api-access-76kn8\") pod \"5f21d3c6-2d63-4a11-943d-79d446f3ba58\" (UID: \"5f21d3c6-2d63-4a11-943d-79d446f3ba58\") " Sep 07 13:18:16.646394 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.646350 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b05aa5e9-207a-4f43-9030-05b6b9f104ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b05aa5e9-207a-4f43-9030-05b6b9f104ca" (UID: "b05aa5e9-207a-4f43-9030-05b6b9f104ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:18:16.646501 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.646485 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b05aa5e9-207a-4f43-9030-05b6b9f104ca-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:18:16.646561 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.646478 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f21d3c6-2d63-4a11-943d-79d446f3ba58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5f21d3c6-2d63-4a11-943d-79d446f3ba58" (UID: "5f21d3c6-2d63-4a11-943d-79d446f3ba58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:18:16.648414 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.648384 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b05aa5e9-207a-4f43-9030-05b6b9f104ca-kube-api-access-mmj98" (OuterVolumeSpecName: "kube-api-access-mmj98") pod "b05aa5e9-207a-4f43-9030-05b6b9f104ca" (UID: "b05aa5e9-207a-4f43-9030-05b6b9f104ca"). InnerVolumeSpecName "kube-api-access-mmj98". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:18:16.648527 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.648418 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f21d3c6-2d63-4a11-943d-79d446f3ba58-kube-api-access-76kn8" (OuterVolumeSpecName: "kube-api-access-76kn8") pod "5f21d3c6-2d63-4a11-943d-79d446f3ba58" (UID: "5f21d3c6-2d63-4a11-943d-79d446f3ba58"). InnerVolumeSpecName "kube-api-access-76kn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:18:16.747912 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.747876 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmj98\" (UniqueName: \"kubernetes.io/projected/b05aa5e9-207a-4f43-9030-05b6b9f104ca-kube-api-access-mmj98\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:18:16.747912 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.747905 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f21d3c6-2d63-4a11-943d-79d446f3ba58-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:18:16.747912 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.747916 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76kn8\" (UniqueName: \"kubernetes.io/projected/5f21d3c6-2d63-4a11-943d-79d446f3ba58-kube-api-access-76kn8\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:18:16.865626 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:16.865592 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:17.154778 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.154689 2576 generic.go:356] "Generic (PLEG): container finished" podID="b05aa5e9-207a-4f43-9030-05b6b9f104ca" containerID="4e953c1056c57cce163381caffe6bbb0a72785340d696029eba5c172875d0f67" exitCode=0 Sep 07 13:18:17.154778 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.154740 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" Sep 07 13:18:17.155049 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.154768 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" event={"ID":"b05aa5e9-207a-4f43-9030-05b6b9f104ca","Type":"ContainerDied","Data":"4e953c1056c57cce163381caffe6bbb0a72785340d696029eba5c172875d0f67"} Sep 07 13:18:17.155049 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.154876 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl" event={"ID":"b05aa5e9-207a-4f43-9030-05b6b9f104ca","Type":"ContainerDied","Data":"9b16431d3468b8ad373c0185d92832b522ab99d7cd65a5c3cb748cdc6c73bf47"} Sep 07 13:18:17.155049 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.154926 2576 scope.go:117] "RemoveContainer" containerID="4e953c1056c57cce163381caffe6bbb0a72785340d696029eba5c172875d0f67" Sep 07 13:18:17.156279 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.156258 2576 generic.go:356] "Generic (PLEG): container finished" podID="5f21d3c6-2d63-4a11-943d-79d446f3ba58" containerID="9a71d2a5eb538ce6e62d71e43d1f2f2f2ba4d2e48e9402fb929722e5256e70e5" exitCode=0 Sep 07 13:18:17.156373 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.156314 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" Sep 07 13:18:17.156373 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.156348 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" event={"ID":"5f21d3c6-2d63-4a11-943d-79d446f3ba58","Type":"ContainerDied","Data":"9a71d2a5eb538ce6e62d71e43d1f2f2f2ba4d2e48e9402fb929722e5256e70e5"} Sep 07 13:18:17.156465 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.156396 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp" event={"ID":"5f21d3c6-2d63-4a11-943d-79d446f3ba58","Type":"ContainerDied","Data":"68a0f1ecf9ff477b2185443341e07854e95e26f92d295db200ef79a4cb1bf869"} Sep 07 13:18:17.165548 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.165530 2576 scope.go:117] "RemoveContainer" containerID="4e953c1056c57cce163381caffe6bbb0a72785340d696029eba5c172875d0f67" Sep 07 13:18:17.165838 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:18:17.165818 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e953c1056c57cce163381caffe6bbb0a72785340d696029eba5c172875d0f67\": container with ID starting with 4e953c1056c57cce163381caffe6bbb0a72785340d696029eba5c172875d0f67 not found: ID does not exist" containerID="4e953c1056c57cce163381caffe6bbb0a72785340d696029eba5c172875d0f67" Sep 07 13:18:17.165906 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.165845 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e953c1056c57cce163381caffe6bbb0a72785340d696029eba5c172875d0f67"} err="failed to get container status \"4e953c1056c57cce163381caffe6bbb0a72785340d696029eba5c172875d0f67\": rpc error: code = NotFound desc = could not find container \"4e953c1056c57cce163381caffe6bbb0a72785340d696029eba5c172875d0f67\": container with ID starting with 4e953c1056c57cce163381caffe6bbb0a72785340d696029eba5c172875d0f67 not found: ID does not exist" Sep 07 13:18:17.165906 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.165862 2576 scope.go:117] "RemoveContainer" containerID="9a71d2a5eb538ce6e62d71e43d1f2f2f2ba4d2e48e9402fb929722e5256e70e5" Sep 07 13:18:17.174456 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.174441 2576 scope.go:117] "RemoveContainer" containerID="9a71d2a5eb538ce6e62d71e43d1f2f2f2ba4d2e48e9402fb929722e5256e70e5" Sep 07 13:18:17.174688 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:18:17.174665 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a71d2a5eb538ce6e62d71e43d1f2f2f2ba4d2e48e9402fb929722e5256e70e5\": container with ID starting with 9a71d2a5eb538ce6e62d71e43d1f2f2f2ba4d2e48e9402fb929722e5256e70e5 not found: ID does not exist" containerID="9a71d2a5eb538ce6e62d71e43d1f2f2f2ba4d2e48e9402fb929722e5256e70e5" Sep 07 13:18:17.174737 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.174693 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a71d2a5eb538ce6e62d71e43d1f2f2f2ba4d2e48e9402fb929722e5256e70e5"} err="failed to get container status \"9a71d2a5eb538ce6e62d71e43d1f2f2f2ba4d2e48e9402fb929722e5256e70e5\": rpc error: code = NotFound desc = could not find container \"9a71d2a5eb538ce6e62d71e43d1f2f2f2ba4d2e48e9402fb929722e5256e70e5\": container with ID starting with 9a71d2a5eb538ce6e62d71e43d1f2f2f2ba4d2e48e9402fb929722e5256e70e5 not found: ID does not exist" Sep 07 13:18:17.184465 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.184443 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl"] Sep 07 13:18:17.196502 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.196480 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ddb9ca-b88b47cb5-qxmzl"] Sep 07 13:18:17.215582 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.215553 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp"] Sep 07 13:18:17.220499 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:17.220476 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ddb9ca-597c5d7466-nfpbp"] Sep 07 13:18:18.483290 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:18.483260 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f21d3c6-2d63-4a11-943d-79d446f3ba58" path="/var/lib/kubelet/pods/5f21d3c6-2d63-4a11-943d-79d446f3ba58/volumes" Sep 07 13:18:18.483661 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:18.483618 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b05aa5e9-207a-4f43-9030-05b6b9f104ca" path="/var/lib/kubelet/pods/b05aa5e9-207a-4f43-9030-05b6b9f104ca/volumes" Sep 07 13:18:18.768458 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:18.768369 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:18.869137 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:18.869104 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:23.744077 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:23.744042 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:23.845588 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:23.845545 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:23.978235 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:23.978195 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:24.063374 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.063343 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:24.418637 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.418555 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn"] Sep 07 13:18:24.418934 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.418922 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b05aa5e9-207a-4f43-9030-05b6b9f104ca" containerName="payload-processing" Sep 07 13:18:24.419014 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.418936 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b05aa5e9-207a-4f43-9030-05b6b9f104ca" containerName="payload-processing" Sep 07 13:18:24.419014 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.418953 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53f13e66-dd38-4e34-a068-c659a3e7d223" containerName="revoke-keys" Sep 07 13:18:24.419014 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.418958 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="53f13e66-dd38-4e34-a068-c659a3e7d223" containerName="revoke-keys" Sep 07 13:18:24.419014 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.418997 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f21d3c6-2d63-4a11-943d-79d446f3ba58" containerName="payload-pre-processing" Sep 07 13:18:24.419014 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.419007 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f21d3c6-2d63-4a11-943d-79d446f3ba58" containerName="payload-pre-processing" Sep 07 13:18:24.419182 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.419092 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b05aa5e9-207a-4f43-9030-05b6b9f104ca" containerName="payload-processing" Sep 07 13:18:24.419182 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.419104 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f21d3c6-2d63-4a11-943d-79d446f3ba58" containerName="payload-pre-processing" Sep 07 13:18:24.419182 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.419111 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="53f13e66-dd38-4e34-a068-c659a3e7d223" containerName="revoke-keys" Sep 07 13:18:24.423749 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.423732 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn" Sep 07 13:18:24.425751 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.425730 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-flngs\"" Sep 07 13:18:24.425880 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.425732 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 13:18:24.430299 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.430277 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn"] Sep 07 13:18:24.528598 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.528559 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b864ac2-d9ab-481b-ac92-e6e94cd9aea1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn\" (UID: \"1b864ac2-d9ab-481b-ac92-e6e94cd9aea1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn" Sep 07 13:18:24.536820 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.536791 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:24.629139 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.629109 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b864ac2-d9ab-481b-ac92-e6e94cd9aea1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn\" (UID: \"1b864ac2-d9ab-481b-ac92-e6e94cd9aea1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn" Sep 07 13:18:24.629723 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.629703 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b864ac2-d9ab-481b-ac92-e6e94cd9aea1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn\" (UID: \"1b864ac2-d9ab-481b-ac92-e6e94cd9aea1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn" Sep 07 13:18:24.736634 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.736550 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn" Sep 07 13:18:24.848118 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:24.848089 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:25.009189 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.009170 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn"] Sep 07 13:18:25.012116 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:18:25.012087 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b864ac2_d9ab_481b_ac92_e6e94cd9aea1.slice/crio-2c7d341e9c4245abfb1db652db38070003874eed6a41019780cade0114179ad6 WatchSource:0}: Error finding container 2c7d341e9c4245abfb1db652db38070003874eed6a41019780cade0114179ad6: Status 404 returned error can't find the container with id 2c7d341e9c4245abfb1db652db38070003874eed6a41019780cade0114179ad6 Sep 07 13:18:25.015627 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.015585 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:25.205813 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.205784 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn" event={"ID":"1b864ac2-d9ab-481b-ac92-e6e94cd9aea1","Type":"ContainerStarted","Data":"2c7d341e9c4245abfb1db652db38070003874eed6a41019780cade0114179ad6"} Sep 07 13:18:25.509525 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.509491 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:25.854161 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.854129 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4"] Sep 07 13:18:25.858063 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.858044 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:25.862114 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.862096 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-ec10ff-openshift-default-dockercfg-44jqc\"" Sep 07 13:18:25.872256 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.872232 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4"] Sep 07 13:18:25.882292 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.882269 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:25.944422 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.944324 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istiod-ca-cert\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:25.944422 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.944394 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-workload-socket\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:25.944651 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.944423 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-workload-certs\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:25.944651 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.944472 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-envoy\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:25.944651 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.944505 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-credential-socket\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:25.944651 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.944548 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kthz4\" (UniqueName: \"kubernetes.io/projected/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-kube-api-access-kthz4\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:25.944651 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.944579 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-podinfo\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:25.944896 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.944687 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-token\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:25.944896 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.944735 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-data\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:25.988301 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:25.988261 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:26.045453 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.045415 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istiod-ca-cert\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.045643 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.045469 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-workload-socket\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.045643 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.045490 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-workload-certs\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.045643 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.045523 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-envoy\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.045643 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.045547 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-credential-socket\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.045643 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.045573 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kthz4\" (UniqueName: \"kubernetes.io/projected/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-kube-api-access-kthz4\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.045903 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.045652 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-podinfo\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.045903 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.045762 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-token\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.045903 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.045803 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-data\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.046086 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.046051 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-credential-socket\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.046299 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.046274 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istiod-ca-cert\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.046299 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.046290 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-workload-socket\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.046539 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.046517 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-workload-certs\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.046685 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.046665 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-data\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.048016 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.047994 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-envoy\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.048301 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.048281 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-podinfo\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.054265 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.054244 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-token\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.054668 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.054650 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kthz4\" (UniqueName: \"kubernetes.io/projected/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-kube-api-access-kthz4\") pod \"e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.168202 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.168169 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:26.214087 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.213994 2576 generic.go:356] "Generic (PLEG): container finished" podID="1b864ac2-d9ab-481b-ac92-e6e94cd9aea1" containerID="7ff36b36a3da840789ffb936418ff011ccf2074982f495e0518f3de9ff1e546a" exitCode=0 Sep 07 13:18:26.214087 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.214059 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn" event={"ID":"1b864ac2-d9ab-481b-ac92-e6e94cd9aea1","Type":"ContainerDied","Data":"7ff36b36a3da840789ffb936418ff011ccf2074982f495e0518f3de9ff1e546a"} Sep 07 13:18:26.462525 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.462493 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4"] Sep 07 13:18:26.463583 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:18:26.463557 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf91ba3a_b68e_49cc_aef0_424e9d17dd98.slice/crio-8074c13b6ede59af889a6ddc629be4ef52c0db825f6c374b3c53bfd271b39361 WatchSource:0}: Error finding container 8074c13b6ede59af889a6ddc629be4ef52c0db825f6c374b3c53bfd271b39361: Status 404 returned error can't find the container with id 8074c13b6ede59af889a6ddc629be4ef52c0db825f6c374b3c53bfd271b39361 Sep 07 13:18:26.466834 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.466797 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:18:26.466918 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.466880 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:18:26.466988 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:26.466919 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:18:27.221269 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:27.221222 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" event={"ID":"bf91ba3a-b68e-49cc-aef0-424e9d17dd98","Type":"ContainerStarted","Data":"2f694e3ce702f5a5a2844267f42904cf9839a20466691a46d19b0cf072a89918"} Sep 07 13:18:27.221770 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:27.221324 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" event={"ID":"bf91ba3a-b68e-49cc-aef0-424e9d17dd98","Type":"ContainerStarted","Data":"8074c13b6ede59af889a6ddc629be4ef52c0db825f6c374b3c53bfd271b39361"} Sep 07 13:18:27.246250 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:27.246167 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" podStartSLOduration=2.246149871 podStartE2EDuration="2.246149871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:18:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:18:27.244439253 +0000 UTC m=+1541.275189025" watchObservedRunningTime="2026-09-07 13:18:27.246149871 +0000 UTC m=+1541.276899714" Sep 07 13:18:27.369724 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:27.369171 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn" Sep 07 13:18:27.460084 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:27.460052 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b864ac2-d9ab-481b-ac92-e6e94cd9aea1-maas-api-service-ca\") pod \"1b864ac2-d9ab-481b-ac92-e6e94cd9aea1\" (UID: \"1b864ac2-d9ab-481b-ac92-e6e94cd9aea1\") " Sep 07 13:18:27.460412 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:27.460388 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b864ac2-d9ab-481b-ac92-e6e94cd9aea1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1b864ac2-d9ab-481b-ac92-e6e94cd9aea1" (UID: "1b864ac2-d9ab-481b-ac92-e6e94cd9aea1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:18:27.561363 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:27.561328 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b864ac2-d9ab-481b-ac92-e6e94cd9aea1-maas-api-service-ca\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:18:28.168958 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:28.168920 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:28.174612 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:28.174588 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:28.228437 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:28.228406 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn" event={"ID":"1b864ac2-d9ab-481b-ac92-e6e94cd9aea1","Type":"ContainerDied","Data":"2c7d341e9c4245abfb1db652db38070003874eed6a41019780cade0114179ad6"} Sep 07 13:18:28.228868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:28.228460 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c7d341e9c4245abfb1db652db38070003874eed6a41019780cade0114179ad6" Sep 07 13:18:28.228868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:28.228411 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn" Sep 07 13:18:28.228868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:28.228845 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:28.229600 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:28.229580 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:29.335586 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:29.335544 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:30.911510 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:30.911472 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:31.019880 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:31.019840 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:31.108377 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:31.108340 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:34.754231 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:34.754200 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:40.510046 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:40.510009 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:40.576469 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:40.576435 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w"] Sep 07 13:18:40.576910 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:40.576884 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" podUID="547e1e0a-513d-4775-8e7a-dd5a545b3b8a" containerName="payload-processing" containerID="cri-o://c84b829a63ab7f35f90d8a822872ffc8655d7c89896005d53f02d50cfc389f1a" gracePeriod=30 Sep 07 13:18:40.621547 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:40.621515 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:40.714500 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:40.714477 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:40.808049 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:40.808006 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:40.826366 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:40.826332 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn"] Sep 07 13:18:40.831843 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:40.831815 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0d2b4b-r7pcn"] Sep 07 13:18:40.904236 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:40.904184 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" Sep 07 13:18:40.933075 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:40.932996 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:41.008323 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.008291 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547e1e0a-513d-4775-8e7a-dd5a545b3b8a-plugins-config-volume\") pod \"547e1e0a-513d-4775-8e7a-dd5a545b3b8a\" (UID: \"547e1e0a-513d-4775-8e7a-dd5a545b3b8a\") " Sep 07 13:18:41.008521 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.008379 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcr8c\" (UniqueName: \"kubernetes.io/projected/547e1e0a-513d-4775-8e7a-dd5a545b3b8a-kube-api-access-rcr8c\") pod \"547e1e0a-513d-4775-8e7a-dd5a545b3b8a\" (UID: \"547e1e0a-513d-4775-8e7a-dd5a545b3b8a\") " Sep 07 13:18:41.008745 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.008713 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/547e1e0a-513d-4775-8e7a-dd5a545b3b8a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "547e1e0a-513d-4775-8e7a-dd5a545b3b8a" (UID: "547e1e0a-513d-4775-8e7a-dd5a545b3b8a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:18:41.010758 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.010727 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/547e1e0a-513d-4775-8e7a-dd5a545b3b8a-kube-api-access-rcr8c" (OuterVolumeSpecName: "kube-api-access-rcr8c") pod "547e1e0a-513d-4775-8e7a-dd5a545b3b8a" (UID: "547e1e0a-513d-4775-8e7a-dd5a545b3b8a"). InnerVolumeSpecName "kube-api-access-rcr8c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:18:41.109438 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.109380 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcr8c\" (UniqueName: \"kubernetes.io/projected/547e1e0a-513d-4775-8e7a-dd5a545b3b8a-kube-api-access-rcr8c\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:18:41.109438 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.109432 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547e1e0a-513d-4775-8e7a-dd5a545b3b8a-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:18:41.254098 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.254060 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:41.306559 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.306531 2576 generic.go:356] "Generic (PLEG): container finished" podID="547e1e0a-513d-4775-8e7a-dd5a545b3b8a" containerID="c84b829a63ab7f35f90d8a822872ffc8655d7c89896005d53f02d50cfc389f1a" exitCode=0 Sep 07 13:18:41.306746 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.306579 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" Sep 07 13:18:41.306746 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.306620 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" event={"ID":"547e1e0a-513d-4775-8e7a-dd5a545b3b8a","Type":"ContainerDied","Data":"c84b829a63ab7f35f90d8a822872ffc8655d7c89896005d53f02d50cfc389f1a"} Sep 07 13:18:41.306746 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.306680 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w" event={"ID":"547e1e0a-513d-4775-8e7a-dd5a545b3b8a","Type":"ContainerDied","Data":"b5aa4c47ff7f1a4dfeb4b5558bdb0b28bf0df4be673889747b28f8d49fe2eae5"} Sep 07 13:18:41.306746 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.306704 2576 scope.go:117] "RemoveContainer" containerID="c84b829a63ab7f35f90d8a822872ffc8655d7c89896005d53f02d50cfc389f1a" Sep 07 13:18:41.315850 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.315831 2576 scope.go:117] "RemoveContainer" containerID="c84b829a63ab7f35f90d8a822872ffc8655d7c89896005d53f02d50cfc389f1a" Sep 07 13:18:41.316118 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:18:41.316100 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c84b829a63ab7f35f90d8a822872ffc8655d7c89896005d53f02d50cfc389f1a\": container with ID starting with c84b829a63ab7f35f90d8a822872ffc8655d7c89896005d53f02d50cfc389f1a not found: ID does not exist" containerID="c84b829a63ab7f35f90d8a822872ffc8655d7c89896005d53f02d50cfc389f1a" Sep 07 13:18:41.316190 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.316124 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c84b829a63ab7f35f90d8a822872ffc8655d7c89896005d53f02d50cfc389f1a"} err="failed to get container status \"c84b829a63ab7f35f90d8a822872ffc8655d7c89896005d53f02d50cfc389f1a\": rpc error: code = NotFound desc = could not find container \"c84b829a63ab7f35f90d8a822872ffc8655d7c89896005d53f02d50cfc389f1a\": container with ID starting with c84b829a63ab7f35f90d8a822872ffc8655d7c89896005d53f02d50cfc389f1a not found: ID does not exist" Sep 07 13:18:41.329516 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.329495 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w"] Sep 07 13:18:41.334539 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:41.334519 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0d2b4b-67b9dc7469-qmj9w"] Sep 07 13:18:42.486658 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:42.486615 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b864ac2-d9ab-481b-ac92-e6e94cd9aea1" path="/var/lib/kubelet/pods/1b864ac2-d9ab-481b-ac92-e6e94cd9aea1/volumes" Sep 07 13:18:42.487195 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:42.487173 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="547e1e0a-513d-4775-8e7a-dd5a545b3b8a" path="/var/lib/kubelet/pods/547e1e0a-513d-4775-8e7a-dd5a545b3b8a/volumes" Sep 07 13:18:48.353888 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:48.353853 2576 generic.go:356] "Generic (PLEG): container finished" podID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerID="2f694e3ce702f5a5a2844267f42904cf9839a20466691a46d19b0cf072a89918" exitCode=0 Sep 07 13:18:48.354556 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:48.353930 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" event={"ID":"bf91ba3a-b68e-49cc-aef0-424e9d17dd98","Type":"ContainerDied","Data":"2f694e3ce702f5a5a2844267f42904cf9839a20466691a46d19b0cf072a89918"} Sep 07 13:18:48.354556 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:48.354376 2576 scope.go:117] "RemoveContainer" containerID="2f694e3ce702f5a5a2844267f42904cf9839a20466691a46d19b0cf072a89918" Sep 07 13:18:48.355566 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:48.355534 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:18:48.355701 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:48.355606 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:18:48.355701 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:48.355641 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 07 13:18:48.384864 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:48.384833 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:48.483068 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:48.483032 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:48.682755 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:48.682669 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:49.064659 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.064623 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c"] Sep 07 13:18:49.065029 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.065015 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b864ac2-d9ab-481b-ac92-e6e94cd9aea1" containerName="revoke-keys" Sep 07 13:18:49.065029 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.065029 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b864ac2-d9ab-481b-ac92-e6e94cd9aea1" containerName="revoke-keys" Sep 07 13:18:49.065150 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.065043 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="547e1e0a-513d-4775-8e7a-dd5a545b3b8a" containerName="payload-processing" Sep 07 13:18:49.065150 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.065049 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="547e1e0a-513d-4775-8e7a-dd5a545b3b8a" containerName="payload-processing" Sep 07 13:18:49.065150 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.065116 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="547e1e0a-513d-4775-8e7a-dd5a545b3b8a" containerName="payload-processing" Sep 07 13:18:49.065150 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.065125 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b864ac2-d9ab-481b-ac92-e6e94cd9aea1" containerName="revoke-keys" Sep 07 13:18:49.075830 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.075801 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c" Sep 07 13:18:49.077854 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.077825 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 13:18:49.078012 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.077927 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-flngs\"" Sep 07 13:18:49.095914 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.095883 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c"] Sep 07 13:18:49.177533 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.177485 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c\" (UID: \"fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c" Sep 07 13:18:49.279749 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.279707 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c\" (UID: \"fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c" Sep 07 13:18:49.280994 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.280955 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c\" (UID: \"fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c" Sep 07 13:18:49.300005 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.298622 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:49.361063 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.360964 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" event={"ID":"bf91ba3a-b68e-49cc-aef0-424e9d17dd98","Type":"ContainerStarted","Data":"5bb874baffe051c64a908517ee9a4a61ce6815d5f288113c02e4e5e3379b79e6"} Sep 07 13:18:49.386871 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.386825 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c" Sep 07 13:18:49.663805 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:49.663780 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c"] Sep 07 13:18:49.666051 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:18:49.666024 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcaa7a8c_6a9c_42fb_a9d1_3e6ff4d30e50.slice/crio-f6f9fea7307821f0a9aa7a8e739ed6319195b71df6de3c6b8b9c83d3d3aeb6e1 WatchSource:0}: Error finding container f6f9fea7307821f0a9aa7a8e739ed6319195b71df6de3c6b8b9c83d3d3aeb6e1: Status 404 returned error can't find the container with id f6f9fea7307821f0a9aa7a8e739ed6319195b71df6de3c6b8b9c83d3d3aeb6e1 Sep 07 13:18:50.168790 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:50.168750 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:50.169550 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:50.169208 2576 patch_prober.go:28] interesting pod/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Sep 07 13:18:50.169550 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:50.169273 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" podUID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Sep 07 13:18:50.373949 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:50.373854 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c" event={"ID":"fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50","Type":"ContainerStarted","Data":"6777f41357ec3bf552d4941f7a0dbf8315db2afa9877918ebf5d499e5ad20789"} Sep 07 13:18:50.374569 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:50.374527 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c" event={"ID":"fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50","Type":"ContainerStarted","Data":"f6f9fea7307821f0a9aa7a8e739ed6319195b71df6de3c6b8b9c83d3d3aeb6e1"} Sep 07 13:18:50.395124 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:50.395070 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c" podStartSLOduration=1.395040118 podStartE2EDuration="1.395040118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:18:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:18:50.394405832 +0000 UTC m=+1564.425155601" watchObservedRunningTime="2026-09-07 13:18:50.395040118 +0000 UTC m=+1564.425789895" Sep 07 13:18:51.169473 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:51.169432 2576 patch_prober.go:28] interesting pod/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Sep 07 13:18:51.169697 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:51.169503 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" podUID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Sep 07 13:18:51.384428 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:51.384388 2576 generic.go:356] "Generic (PLEG): container finished" podID="fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50" containerID="6777f41357ec3bf552d4941f7a0dbf8315db2afa9877918ebf5d499e5ad20789" exitCode=0 Sep 07 13:18:51.384910 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:51.384497 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c" event={"ID":"fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50","Type":"ContainerDied","Data":"6777f41357ec3bf552d4941f7a0dbf8315db2afa9877918ebf5d499e5ad20789"} Sep 07 13:18:51.474889 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:51.474793 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:52.169015 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:52.168955 2576 patch_prober.go:28] interesting pod/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Sep 07 13:18:52.169328 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:52.169298 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" podUID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Sep 07 13:18:52.556265 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:52.556237 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c" Sep 07 13:18:52.620151 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:52.620124 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50-maas-api-service-ca\") pod \"fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50\" (UID: \"fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50\") " Sep 07 13:18:52.620480 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:52.620455 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50" (UID: "fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:18:52.721827 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:52.721734 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50-maas-api-service-ca\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:18:53.169651 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:53.169611 2576 patch_prober.go:28] interesting pod/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Sep 07 13:18:53.169910 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:53.169884 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" podUID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Sep 07 13:18:53.400578 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:53.400549 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c" Sep 07 13:18:53.400739 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:53.400584 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c" event={"ID":"fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50","Type":"ContainerDied","Data":"f6f9fea7307821f0a9aa7a8e739ed6319195b71df6de3c6b8b9c83d3d3aeb6e1"} Sep 07 13:18:53.400739 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:53.400636 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6f9fea7307821f0a9aa7a8e739ed6319195b71df6de3c6b8b9c83d3d3aeb6e1" Sep 07 13:18:54.169544 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:54.169493 2576 patch_prober.go:28] interesting pod/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Sep 07 13:18:54.170089 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:54.169568 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" podUID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Sep 07 13:18:54.361400 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:54.361358 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:55.173068 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:55.173038 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:55.173472 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:55.173322 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:55.174087 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:55.174068 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:18:55.213256 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:55.213222 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:55.481892 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:55.481804 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:56.068434 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:56.068373 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:58.855177 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:58.855147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:58.960986 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:58.960941 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:18:59.271673 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:18:59.271591 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:00.303376 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:00.303334 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:00.378807 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:00.378775 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:00.567446 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:00.567355 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:00.850773 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:00.850686 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:00.952525 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:00.952492 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:00.984104 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:00.984070 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv"] Sep 07 13:19:00.984459 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:00.984448 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50" containerName="revoke-keys" Sep 07 13:19:00.984513 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:00.984462 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50" containerName="revoke-keys" Sep 07 13:19:00.984549 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:00.984542 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50" containerName="revoke-keys" Sep 07 13:19:00.988942 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:00.988918 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" Sep 07 13:19:00.990888 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:00.990855 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-688b44-dockercfg-bzhdn\"" Sep 07 13:19:00.990888 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:00.990879 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-688b44\"" Sep 07 13:19:00.998994 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:00.998953 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv"] Sep 07 13:19:01.107010 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.106741 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e960068-c20f-4725-8006-f32ca3faf414-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv\" (UID: \"9e960068-c20f-4725-8006-f32ca3faf414\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" Sep 07 13:19:01.107010 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.106817 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmzw9\" (UniqueName: \"kubernetes.io/projected/9e960068-c20f-4725-8006-f32ca3faf414-kube-api-access-nmzw9\") pod \"payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv\" (UID: \"9e960068-c20f-4725-8006-f32ca3faf414\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" Sep 07 13:19:01.181855 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.181821 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:01.208419 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.208389 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmzw9\" (UniqueName: \"kubernetes.io/projected/9e960068-c20f-4725-8006-f32ca3faf414-kube-api-access-nmzw9\") pod \"payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv\" (UID: \"9e960068-c20f-4725-8006-f32ca3faf414\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" Sep 07 13:19:01.208603 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.208552 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e960068-c20f-4725-8006-f32ca3faf414-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv\" (UID: \"9e960068-c20f-4725-8006-f32ca3faf414\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" Sep 07 13:19:01.209333 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.209305 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e960068-c20f-4725-8006-f32ca3faf414-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv\" (UID: \"9e960068-c20f-4725-8006-f32ca3faf414\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" Sep 07 13:19:01.218305 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.218284 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmzw9\" (UniqueName: \"kubernetes.io/projected/9e960068-c20f-4725-8006-f32ca3faf414-kube-api-access-nmzw9\") pod \"payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv\" (UID: \"9e960068-c20f-4725-8006-f32ca3faf414\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" Sep 07 13:19:01.272852 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.272818 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:01.302003 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.301961 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" Sep 07 13:19:01.406251 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.406164 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c"] Sep 07 13:19:01.408537 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.408514 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bc4b05-5448c"] Sep 07 13:19:01.453931 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.453900 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:01.520862 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.520828 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:01.590474 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.590450 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv"] Sep 07 13:19:01.593173 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:19:01.593146 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e960068_c20f_4725_8006_f32ca3faf414.slice/crio-8b0828eca787c979d7d71f3614fd21dd5ca8baea479fe6f889c44ceeda064601 WatchSource:0}: Error finding container 8b0828eca787c979d7d71f3614fd21dd5ca8baea479fe6f889c44ceeda064601: Status 404 returned error can't find the container with id 8b0828eca787c979d7d71f3614fd21dd5ca8baea479fe6f889c44ceeda064601 Sep 07 13:19:01.814639 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:01.814599 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:02.453623 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:02.453590 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" event={"ID":"9e960068-c20f-4725-8006-f32ca3faf414","Type":"ContainerStarted","Data":"efeb608523acfa5a5687530e99919de199d2b09139b84b6407e5423beca51c53"} Sep 07 13:19:02.454001 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:02.453743 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" Sep 07 13:19:02.454001 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:02.453784 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" event={"ID":"9e960068-c20f-4725-8006-f32ca3faf414","Type":"ContainerStarted","Data":"8b0828eca787c979d7d71f3614fd21dd5ca8baea479fe6f889c44ceeda064601"} Sep 07 13:19:02.473265 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:02.473212 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" podStartSLOduration=2.473195294 podStartE2EDuration="2.473195294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:19:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:19:02.470798199 +0000 UTC m=+1576.501547987" watchObservedRunningTime="2026-09-07 13:19:02.473195294 +0000 UTC m=+1576.503945065" Sep 07 13:19:02.483940 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:02.483903 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50" path="/var/lib/kubelet/pods/fcaa7a8c-6a9c-42fb-a9d1-3e6ff4d30e50/volumes" Sep 07 13:19:02.958516 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:02.958477 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:08.596656 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:08.596619 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:08.761429 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:08.761394 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:08.928801 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:08.928702 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:09.487230 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:09.487186 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:13.461313 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:13.461286 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" Sep 07 13:19:19.865854 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:19.865818 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl"] Sep 07 13:19:19.868289 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:19.868235 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-45d721-29813115-d6kwl"] Sep 07 13:19:19.868465 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:19.868388 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:19.899598 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:19.899566 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5"] Sep 07 13:19:19.899836 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:19.899809 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" podUID="8e135966-53cb-4eca-917d-efcba7c2595f" containerName="payload-processing" containerID="cri-o://76e91b7ec74def4ced47acd5f6ddddcd2448f8f1854dba1dfc2ec5917d4bdc3e" gracePeriod=30 Sep 07 13:19:19.969881 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:19.969841 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:20.070523 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.070485 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:20.162403 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.162374 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" Sep 07 13:19:20.169730 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.169702 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:20.299181 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.299150 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs7t4\" (UniqueName: \"kubernetes.io/projected/8e135966-53cb-4eca-917d-efcba7c2595f-kube-api-access-qs7t4\") pod \"8e135966-53cb-4eca-917d-efcba7c2595f\" (UID: \"8e135966-53cb-4eca-917d-efcba7c2595f\") " Sep 07 13:19:20.299389 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.299216 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e135966-53cb-4eca-917d-efcba7c2595f-plugins-config-volume\") pod \"8e135966-53cb-4eca-917d-efcba7c2595f\" (UID: \"8e135966-53cb-4eca-917d-efcba7c2595f\") " Sep 07 13:19:20.299558 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.299534 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e135966-53cb-4eca-917d-efcba7c2595f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e135966-53cb-4eca-917d-efcba7c2595f" (UID: "8e135966-53cb-4eca-917d-efcba7c2595f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:19:20.301462 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.301433 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e135966-53cb-4eca-917d-efcba7c2595f-kube-api-access-qs7t4" (OuterVolumeSpecName: "kube-api-access-qs7t4") pod "8e135966-53cb-4eca-917d-efcba7c2595f" (UID: "8e135966-53cb-4eca-917d-efcba7c2595f"). InnerVolumeSpecName "kube-api-access-qs7t4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:19:20.399946 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.399863 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e135966-53cb-4eca-917d-efcba7c2595f-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:19:20.399946 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.399898 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs7t4\" (UniqueName: \"kubernetes.io/projected/8e135966-53cb-4eca-917d-efcba7c2595f-kube-api-access-qs7t4\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:19:20.483149 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.483114 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45207102-69f0-4c12-9865-b4e470feaf8b" path="/var/lib/kubelet/pods/45207102-69f0-4c12-9865-b4e470feaf8b/volumes" Sep 07 13:19:20.509031 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.509003 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:20.557918 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.557885 2576 generic.go:356] "Generic (PLEG): container finished" podID="8e135966-53cb-4eca-917d-efcba7c2595f" containerID="76e91b7ec74def4ced47acd5f6ddddcd2448f8f1854dba1dfc2ec5917d4bdc3e" exitCode=0 Sep 07 13:19:20.558106 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.557931 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" Sep 07 13:19:20.558106 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.557987 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" event={"ID":"8e135966-53cb-4eca-917d-efcba7c2595f","Type":"ContainerDied","Data":"76e91b7ec74def4ced47acd5f6ddddcd2448f8f1854dba1dfc2ec5917d4bdc3e"} Sep 07 13:19:20.558106 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.558045 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5" event={"ID":"8e135966-53cb-4eca-917d-efcba7c2595f","Type":"ContainerDied","Data":"f718395397b0f754809433a4666b2df69aa89da6e3a6b6bb30e5e6bbadc583a9"} Sep 07 13:19:20.558106 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.558075 2576 scope.go:117] "RemoveContainer" containerID="76e91b7ec74def4ced47acd5f6ddddcd2448f8f1854dba1dfc2ec5917d4bdc3e" Sep 07 13:19:20.567295 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.567279 2576 scope.go:117] "RemoveContainer" containerID="76e91b7ec74def4ced47acd5f6ddddcd2448f8f1854dba1dfc2ec5917d4bdc3e" Sep 07 13:19:20.567578 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:19:20.567555 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76e91b7ec74def4ced47acd5f6ddddcd2448f8f1854dba1dfc2ec5917d4bdc3e\": container with ID starting with 76e91b7ec74def4ced47acd5f6ddddcd2448f8f1854dba1dfc2ec5917d4bdc3e not found: ID does not exist" containerID="76e91b7ec74def4ced47acd5f6ddddcd2448f8f1854dba1dfc2ec5917d4bdc3e" Sep 07 13:19:20.567667 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.567584 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76e91b7ec74def4ced47acd5f6ddddcd2448f8f1854dba1dfc2ec5917d4bdc3e"} err="failed to get container status \"76e91b7ec74def4ced47acd5f6ddddcd2448f8f1854dba1dfc2ec5917d4bdc3e\": rpc error: code = NotFound desc = could not find container \"76e91b7ec74def4ced47acd5f6ddddcd2448f8f1854dba1dfc2ec5917d4bdc3e\": container with ID starting with 76e91b7ec74def4ced47acd5f6ddddcd2448f8f1854dba1dfc2ec5917d4bdc3e not found: ID does not exist" Sep 07 13:19:20.575588 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.575566 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5"] Sep 07 13:19:20.579750 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:20.579730 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-45d721-56794bfd9-jkdx5"] Sep 07 13:19:21.272759 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:21.272725 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:22.482393 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:22.482357 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e135966-53cb-4eca-917d-efcba7c2595f" path="/var/lib/kubelet/pods/8e135966-53cb-4eca-917d-efcba7c2595f/volumes" Sep 07 13:19:23.938491 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:23.938448 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:27.629714 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:27.629676 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:27.729852 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:27.729822 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:27.905390 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:27.905282 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw"] Sep 07 13:19:27.905591 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:27.905554 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" podUID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerName="istio-proxy" containerID="cri-o://afa412edda94268678da28feb7480b8d682e9d81509e592ff34d229893b48833" gracePeriod=30 Sep 07 13:19:27.929661 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:27.929616 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:19:30.296906 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:30.296864 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9xpdc_46f3ff03-1092-43c3-97cf-7c3fdec4dd71/payload-processing/0.log" Sep 07 13:19:30.427720 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:30.427691 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv_9e960068-c20f-4725-8006-f32ca3faf414/payload-processing/0.log" Sep 07 13:19:33.171399 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.171338 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:19:33.226029 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.225990 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-credential-socket\") pod \"3c45411e-d304-461f-aef1-f6ac915b9a8f\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " Sep 07 13:19:33.226202 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226036 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-token\") pod \"3c45411e-d304-461f-aef1-f6ac915b9a8f\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " Sep 07 13:19:33.226202 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226086 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-workload-socket\") pod \"3c45411e-d304-461f-aef1-f6ac915b9a8f\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " Sep 07 13:19:33.226202 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226122 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-workload-certs\") pod \"3c45411e-d304-461f-aef1-f6ac915b9a8f\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " Sep 07 13:19:33.226202 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226152 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-envoy\") pod \"3c45411e-d304-461f-aef1-f6ac915b9a8f\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " Sep 07 13:19:33.226202 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226193 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c45411e-d304-461f-aef1-f6ac915b9a8f-istiod-ca-cert\") pod \"3c45411e-d304-461f-aef1-f6ac915b9a8f\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " Sep 07 13:19:33.226464 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226233 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-data\") pod \"3c45411e-d304-461f-aef1-f6ac915b9a8f\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " Sep 07 13:19:33.226464 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226281 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c45411e-d304-461f-aef1-f6ac915b9a8f" (UID: "3c45411e-d304-461f-aef1-f6ac915b9a8f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:19:33.226464 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226301 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vh9n\" (UniqueName: \"kubernetes.io/projected/3c45411e-d304-461f-aef1-f6ac915b9a8f-kube-api-access-2vh9n\") pod \"3c45411e-d304-461f-aef1-f6ac915b9a8f\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " Sep 07 13:19:33.226464 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226359 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-podinfo\") pod \"3c45411e-d304-461f-aef1-f6ac915b9a8f\" (UID: \"3c45411e-d304-461f-aef1-f6ac915b9a8f\") " Sep 07 13:19:33.226464 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226385 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c45411e-d304-461f-aef1-f6ac915b9a8f" (UID: "3c45411e-d304-461f-aef1-f6ac915b9a8f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:19:33.226699 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226499 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c45411e-d304-461f-aef1-f6ac915b9a8f" (UID: "3c45411e-d304-461f-aef1-f6ac915b9a8f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:19:33.226699 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226510 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c45411e-d304-461f-aef1-f6ac915b9a8f" (UID: "3c45411e-d304-461f-aef1-f6ac915b9a8f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:19:33.226699 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226613 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c45411e-d304-461f-aef1-f6ac915b9a8f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c45411e-d304-461f-aef1-f6ac915b9a8f" (UID: "3c45411e-d304-461f-aef1-f6ac915b9a8f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:19:33.226699 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226680 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-credential-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:19:33.226699 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226696 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-workload-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:19:33.226955 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226710 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-workload-certs\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:19:33.226955 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226722 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c45411e-d304-461f-aef1-f6ac915b9a8f-istiod-ca-cert\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:19:33.226955 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.226734 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-data\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:19:33.228401 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.228370 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c45411e-d304-461f-aef1-f6ac915b9a8f" (UID: "3c45411e-d304-461f-aef1-f6ac915b9a8f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:19:33.228514 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.228434 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c45411e-d304-461f-aef1-f6ac915b9a8f-kube-api-access-2vh9n" (OuterVolumeSpecName: "kube-api-access-2vh9n") pod "3c45411e-d304-461f-aef1-f6ac915b9a8f" (UID: "3c45411e-d304-461f-aef1-f6ac915b9a8f"). InnerVolumeSpecName "kube-api-access-2vh9n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:19:33.228957 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.228932 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c45411e-d304-461f-aef1-f6ac915b9a8f" (UID: "3c45411e-d304-461f-aef1-f6ac915b9a8f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:19:33.228957 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.228939 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c45411e-d304-461f-aef1-f6ac915b9a8f" (UID: "3c45411e-d304-461f-aef1-f6ac915b9a8f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 13:19:33.327616 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.327588 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vh9n\" (UniqueName: \"kubernetes.io/projected/3c45411e-d304-461f-aef1-f6ac915b9a8f-kube-api-access-2vh9n\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:19:33.327616 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.327614 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-podinfo\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:19:33.327616 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.327624 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-token\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:19:33.327829 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.327632 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c45411e-d304-461f-aef1-f6ac915b9a8f-istio-envoy\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:19:33.634435 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.634404 2576 generic.go:356] "Generic (PLEG): container finished" podID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerID="afa412edda94268678da28feb7480b8d682e9d81509e592ff34d229893b48833" exitCode=0 Sep 07 13:19:33.634617 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.634457 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" event={"ID":"3c45411e-d304-461f-aef1-f6ac915b9a8f","Type":"ContainerDied","Data":"afa412edda94268678da28feb7480b8d682e9d81509e592ff34d229893b48833"} Sep 07 13:19:33.634617 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.634478 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" Sep 07 13:19:33.634617 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.634499 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw" event={"ID":"3c45411e-d304-461f-aef1-f6ac915b9a8f","Type":"ContainerDied","Data":"6bdd4d9416204e7b08e75c1b80b926b3d90222abb00e48aa42047e331cea043a"} Sep 07 13:19:33.634617 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.634521 2576 scope.go:117] "RemoveContainer" containerID="afa412edda94268678da28feb7480b8d682e9d81509e592ff34d229893b48833" Sep 07 13:19:33.644121 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.644056 2576 scope.go:117] "RemoveContainer" containerID="aa5c4ed0f10dc1cd75db15114b8fb70398ceff0d5fe6a6e38b985fd33adb4735" Sep 07 13:19:33.652787 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.652772 2576 scope.go:117] "RemoveContainer" containerID="afa412edda94268678da28feb7480b8d682e9d81509e592ff34d229893b48833" Sep 07 13:19:33.653060 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:19:33.653044 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afa412edda94268678da28feb7480b8d682e9d81509e592ff34d229893b48833\": container with ID starting with afa412edda94268678da28feb7480b8d682e9d81509e592ff34d229893b48833 not found: ID does not exist" containerID="afa412edda94268678da28feb7480b8d682e9d81509e592ff34d229893b48833" Sep 07 13:19:33.653115 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.653067 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afa412edda94268678da28feb7480b8d682e9d81509e592ff34d229893b48833"} err="failed to get container status \"afa412edda94268678da28feb7480b8d682e9d81509e592ff34d229893b48833\": rpc error: code = NotFound desc = could not find container \"afa412edda94268678da28feb7480b8d682e9d81509e592ff34d229893b48833\": container with ID starting with afa412edda94268678da28feb7480b8d682e9d81509e592ff34d229893b48833 not found: ID does not exist" Sep 07 13:19:33.653115 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.653082 2576 scope.go:117] "RemoveContainer" containerID="aa5c4ed0f10dc1cd75db15114b8fb70398ceff0d5fe6a6e38b985fd33adb4735" Sep 07 13:19:33.653324 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:19:33.653306 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa5c4ed0f10dc1cd75db15114b8fb70398ceff0d5fe6a6e38b985fd33adb4735\": container with ID starting with aa5c4ed0f10dc1cd75db15114b8fb70398ceff0d5fe6a6e38b985fd33adb4735 not found: ID does not exist" containerID="aa5c4ed0f10dc1cd75db15114b8fb70398ceff0d5fe6a6e38b985fd33adb4735" Sep 07 13:19:33.653358 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.653331 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa5c4ed0f10dc1cd75db15114b8fb70398ceff0d5fe6a6e38b985fd33adb4735"} err="failed to get container status \"aa5c4ed0f10dc1cd75db15114b8fb70398ceff0d5fe6a6e38b985fd33adb4735\": rpc error: code = NotFound desc = could not find container \"aa5c4ed0f10dc1cd75db15114b8fb70398ceff0d5fe6a6e38b985fd33adb4735\": container with ID starting with aa5c4ed0f10dc1cd75db15114b8fb70398ceff0d5fe6a6e38b985fd33adb4735 not found: ID does not exist" Sep 07 13:19:33.656888 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.656850 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw"] Sep 07 13:19:33.658842 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:33.658824 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-45d721-openshift-default-69b8fb87b6-472rw"] Sep 07 13:19:34.482294 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:34.482261 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c45411e-d304-461f-aef1-f6ac915b9a8f" path="/var/lib/kubelet/pods/3c45411e-d304-461f-aef1-f6ac915b9a8f/volumes" Sep 07 13:19:38.542089 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:19:38.542002 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:01.263836 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:01.263797 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9xpdc_46f3ff03-1092-43c3-97cf-7c3fdec4dd71/payload-processing/0.log" Sep 07 13:20:01.400333 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:01.400303 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv_9e960068-c20f-4725-8006-f32ca3faf414/payload-processing/0.log" Sep 07 13:20:01.726398 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:01.726366 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:02.054939 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:02.054909 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:02.416044 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:02.415921 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:05.344984 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:05.344933 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:05.805112 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:05.805081 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:06.572259 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:06.572212 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:06.734628 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:06.734599 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:06.863854 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:06.863761 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:07.306113 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.306077 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:07.367345 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.367316 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d"] Sep 07 13:20:07.367704 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.367692 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e135966-53cb-4eca-917d-efcba7c2595f" containerName="payload-processing" Sep 07 13:20:07.367748 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.367706 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e135966-53cb-4eca-917d-efcba7c2595f" containerName="payload-processing" Sep 07 13:20:07.367748 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.367717 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerName="istio-proxy" Sep 07 13:20:07.367748 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.367723 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerName="istio-proxy" Sep 07 13:20:07.367748 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.367737 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerName="istio-proxy" Sep 07 13:20:07.367748 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.367743 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerName="istio-proxy" Sep 07 13:20:07.367903 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.367806 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerName="istio-proxy" Sep 07 13:20:07.367903 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.367816 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c45411e-d304-461f-aef1-f6ac915b9a8f" containerName="istio-proxy" Sep 07 13:20:07.367903 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.367822 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e135966-53cb-4eca-917d-efcba7c2595f" containerName="payload-processing" Sep 07 13:20:07.372164 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.372145 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" Sep 07 13:20:07.373921 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.373900 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-a2a8d5\"" Sep 07 13:20:07.381999 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.381949 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d"] Sep 07 13:20:07.385019 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.384995 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:07.438000 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.437934 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85"] Sep 07 13:20:07.441482 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.441466 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" Sep 07 13:20:07.443446 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.443428 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-a2a8d5-dockercfg-kk9g7\"" Sep 07 13:20:07.451464 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.451441 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629f4f87-2c8c-4368-b79e-db94bf347076-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85\" (UID: \"629f4f87-2c8c-4368-b79e-db94bf347076\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" Sep 07 13:20:07.451612 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.451488 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c21e7eb2-8404-4a30-99b5-997c0e8abb54-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d\" (UID: \"c21e7eb2-8404-4a30-99b5-997c0e8abb54\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" Sep 07 13:20:07.451612 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.451552 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdbxn\" (UniqueName: \"kubernetes.io/projected/c21e7eb2-8404-4a30-99b5-997c0e8abb54-kube-api-access-vdbxn\") pod \"payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d\" (UID: \"c21e7eb2-8404-4a30-99b5-997c0e8abb54\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" Sep 07 13:20:07.451722 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.451619 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmgf8\" (UniqueName: \"kubernetes.io/projected/629f4f87-2c8c-4368-b79e-db94bf347076-kube-api-access-tmgf8\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85\" (UID: \"629f4f87-2c8c-4368-b79e-db94bf347076\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" Sep 07 13:20:07.453890 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.453865 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85"] Sep 07 13:20:07.552752 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.552718 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629f4f87-2c8c-4368-b79e-db94bf347076-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85\" (UID: \"629f4f87-2c8c-4368-b79e-db94bf347076\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" Sep 07 13:20:07.552982 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.552779 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c21e7eb2-8404-4a30-99b5-997c0e8abb54-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d\" (UID: \"c21e7eb2-8404-4a30-99b5-997c0e8abb54\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" Sep 07 13:20:07.552982 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.552830 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdbxn\" (UniqueName: \"kubernetes.io/projected/c21e7eb2-8404-4a30-99b5-997c0e8abb54-kube-api-access-vdbxn\") pod \"payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d\" (UID: \"c21e7eb2-8404-4a30-99b5-997c0e8abb54\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" Sep 07 13:20:07.552982 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.552872 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmgf8\" (UniqueName: \"kubernetes.io/projected/629f4f87-2c8c-4368-b79e-db94bf347076-kube-api-access-tmgf8\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85\" (UID: \"629f4f87-2c8c-4368-b79e-db94bf347076\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" Sep 07 13:20:07.553719 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.553692 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629f4f87-2c8c-4368-b79e-db94bf347076-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85\" (UID: \"629f4f87-2c8c-4368-b79e-db94bf347076\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" Sep 07 13:20:07.553770 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.553723 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c21e7eb2-8404-4a30-99b5-997c0e8abb54-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d\" (UID: \"c21e7eb2-8404-4a30-99b5-997c0e8abb54\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" Sep 07 13:20:07.561715 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.561647 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmgf8\" (UniqueName: \"kubernetes.io/projected/629f4f87-2c8c-4368-b79e-db94bf347076-kube-api-access-tmgf8\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85\" (UID: \"629f4f87-2c8c-4368-b79e-db94bf347076\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" Sep 07 13:20:07.561715 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.561653 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdbxn\" (UniqueName: \"kubernetes.io/projected/c21e7eb2-8404-4a30-99b5-997c0e8abb54-kube-api-access-vdbxn\") pod \"payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d\" (UID: \"c21e7eb2-8404-4a30-99b5-997c0e8abb54\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" Sep 07 13:20:07.683339 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.683303 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" Sep 07 13:20:07.751651 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.751612 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" Sep 07 13:20:07.971098 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:07.971071 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d"] Sep 07 13:20:07.973251 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:20:07.973222 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc21e7eb2_8404_4a30_99b5_997c0e8abb54.slice/crio-637417ca5726a6d9c87ddb9761b2d0b52e26db0b1f9fc36de40b3884b78666a5 WatchSource:0}: Error finding container 637417ca5726a6d9c87ddb9761b2d0b52e26db0b1f9fc36de40b3884b78666a5: Status 404 returned error can't find the container with id 637417ca5726a6d9c87ddb9761b2d0b52e26db0b1f9fc36de40b3884b78666a5 Sep 07 13:20:08.042094 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:08.042071 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85"] Sep 07 13:20:08.044765 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:20:08.044735 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod629f4f87_2c8c_4368_b79e_db94bf347076.slice/crio-bd2205f57098ed7c3955bdaff4d9265eb6f605f3cc8ed60d7deb06693cd12f5f WatchSource:0}: Error finding container bd2205f57098ed7c3955bdaff4d9265eb6f605f3cc8ed60d7deb06693cd12f5f: Status 404 returned error can't find the container with id bd2205f57098ed7c3955bdaff4d9265eb6f605f3cc8ed60d7deb06693cd12f5f Sep 07 13:20:08.834823 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:08.834779 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" event={"ID":"c21e7eb2-8404-4a30-99b5-997c0e8abb54","Type":"ContainerStarted","Data":"3bb5ada7204a14796869ce6314290d3296485be2d58903102d78de4c0c24d734"} Sep 07 13:20:08.835322 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:08.834848 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" Sep 07 13:20:08.835322 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:08.834872 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" event={"ID":"c21e7eb2-8404-4a30-99b5-997c0e8abb54","Type":"ContainerStarted","Data":"637417ca5726a6d9c87ddb9761b2d0b52e26db0b1f9fc36de40b3884b78666a5"} Sep 07 13:20:08.836372 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:08.836344 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" event={"ID":"629f4f87-2c8c-4368-b79e-db94bf347076","Type":"ContainerStarted","Data":"8b79b5a1f881850541a7a5c2d1b2580463f04c2d10e40aa2fbd6f592a2be8bf0"} Sep 07 13:20:08.836489 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:08.836387 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" event={"ID":"629f4f87-2c8c-4368-b79e-db94bf347076","Type":"ContainerStarted","Data":"bd2205f57098ed7c3955bdaff4d9265eb6f605f3cc8ed60d7deb06693cd12f5f"} Sep 07 13:20:08.836489 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:08.836416 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" Sep 07 13:20:08.857722 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:08.857669 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" podStartSLOduration=1.8576494270000001 podStartE2EDuration="1.857649427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:20:08.85761854 +0000 UTC m=+1642.888368331" watchObservedRunningTime="2026-09-07 13:20:08.857649427 +0000 UTC m=+1642.888399200" Sep 07 13:20:08.880657 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:08.880589 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" podStartSLOduration=1.880569409 podStartE2EDuration="1.880569409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:20:08.879691053 +0000 UTC m=+1642.910440847" watchObservedRunningTime="2026-09-07 13:20:08.880569409 +0000 UTC m=+1642.911319177" Sep 07 13:20:19.844439 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:19.844410 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" Sep 07 13:20:19.844820 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:19.844475 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" Sep 07 13:20:30.478468 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:30.478432 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:37.128574 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.128531 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:37.167160 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.167118 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d"] Sep 07 13:20:37.167418 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.167388 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" podUID="c21e7eb2-8404-4a30-99b5-997c0e8abb54" containerName="payload-processing" containerID="cri-o://3bb5ada7204a14796869ce6314290d3296485be2d58903102d78de4c0c24d734" gracePeriod=30 Sep 07 13:20:37.173195 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.173166 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85"] Sep 07 13:20:37.173426 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.173398 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" podUID="629f4f87-2c8c-4368-b79e-db94bf347076" containerName="payload-pre-processing" containerID="cri-o://8b79b5a1f881850541a7a5c2d1b2580463f04c2d10e40aa2fbd6f592a2be8bf0" gracePeriod=30 Sep 07 13:20:37.230638 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.230603 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:37.382352 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.382102 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:37.431032 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.431001 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" Sep 07 13:20:37.472858 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.472829 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" Sep 07 13:20:37.548250 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.548216 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c21e7eb2-8404-4a30-99b5-997c0e8abb54-plugins-config-volume\") pod \"c21e7eb2-8404-4a30-99b5-997c0e8abb54\" (UID: \"c21e7eb2-8404-4a30-99b5-997c0e8abb54\") " Sep 07 13:20:37.548425 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.548255 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdbxn\" (UniqueName: \"kubernetes.io/projected/c21e7eb2-8404-4a30-99b5-997c0e8abb54-kube-api-access-vdbxn\") pod \"c21e7eb2-8404-4a30-99b5-997c0e8abb54\" (UID: \"c21e7eb2-8404-4a30-99b5-997c0e8abb54\") " Sep 07 13:20:37.548678 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.548655 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c21e7eb2-8404-4a30-99b5-997c0e8abb54-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c21e7eb2-8404-4a30-99b5-997c0e8abb54" (UID: "c21e7eb2-8404-4a30-99b5-997c0e8abb54"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:20:37.550585 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.550560 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c21e7eb2-8404-4a30-99b5-997c0e8abb54-kube-api-access-vdbxn" (OuterVolumeSpecName: "kube-api-access-vdbxn") pod "c21e7eb2-8404-4a30-99b5-997c0e8abb54" (UID: "c21e7eb2-8404-4a30-99b5-997c0e8abb54"). InnerVolumeSpecName "kube-api-access-vdbxn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:20:37.591748 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.591712 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:37.648961 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.648882 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmgf8\" (UniqueName: \"kubernetes.io/projected/629f4f87-2c8c-4368-b79e-db94bf347076-kube-api-access-tmgf8\") pod \"629f4f87-2c8c-4368-b79e-db94bf347076\" (UID: \"629f4f87-2c8c-4368-b79e-db94bf347076\") " Sep 07 13:20:37.648961 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.648916 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629f4f87-2c8c-4368-b79e-db94bf347076-plugins-config-volume\") pod \"629f4f87-2c8c-4368-b79e-db94bf347076\" (UID: \"629f4f87-2c8c-4368-b79e-db94bf347076\") " Sep 07 13:20:37.649315 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.649291 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/629f4f87-2c8c-4368-b79e-db94bf347076-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "629f4f87-2c8c-4368-b79e-db94bf347076" (UID: "629f4f87-2c8c-4368-b79e-db94bf347076"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:20:37.649479 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.649460 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdbxn\" (UniqueName: \"kubernetes.io/projected/c21e7eb2-8404-4a30-99b5-997c0e8abb54-kube-api-access-vdbxn\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:20:37.649551 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.649481 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/629f4f87-2c8c-4368-b79e-db94bf347076-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:20:37.649551 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.649490 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c21e7eb2-8404-4a30-99b5-997c0e8abb54-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:20:37.651146 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.651122 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/629f4f87-2c8c-4368-b79e-db94bf347076-kube-api-access-tmgf8" (OuterVolumeSpecName: "kube-api-access-tmgf8") pod "629f4f87-2c8c-4368-b79e-db94bf347076" (UID: "629f4f87-2c8c-4368-b79e-db94bf347076"). InnerVolumeSpecName "kube-api-access-tmgf8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:20:37.668177 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.668143 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:37.750501 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.750470 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmgf8\" (UniqueName: \"kubernetes.io/projected/629f4f87-2c8c-4368-b79e-db94bf347076-kube-api-access-tmgf8\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:20:37.845106 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.845073 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:37.986812 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.986720 2576 generic.go:356] "Generic (PLEG): container finished" podID="c21e7eb2-8404-4a30-99b5-997c0e8abb54" containerID="3bb5ada7204a14796869ce6314290d3296485be2d58903102d78de4c0c24d734" exitCode=0 Sep 07 13:20:37.986812 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.986771 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" Sep 07 13:20:37.987049 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.986806 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" event={"ID":"c21e7eb2-8404-4a30-99b5-997c0e8abb54","Type":"ContainerDied","Data":"3bb5ada7204a14796869ce6314290d3296485be2d58903102d78de4c0c24d734"} Sep 07 13:20:37.987049 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.986864 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d" event={"ID":"c21e7eb2-8404-4a30-99b5-997c0e8abb54","Type":"ContainerDied","Data":"637417ca5726a6d9c87ddb9761b2d0b52e26db0b1f9fc36de40b3884b78666a5"} Sep 07 13:20:37.987049 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.986897 2576 scope.go:117] "RemoveContainer" containerID="3bb5ada7204a14796869ce6314290d3296485be2d58903102d78de4c0c24d734" Sep 07 13:20:37.988299 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.988279 2576 generic.go:356] "Generic (PLEG): container finished" podID="629f4f87-2c8c-4368-b79e-db94bf347076" containerID="8b79b5a1f881850541a7a5c2d1b2580463f04c2d10e40aa2fbd6f592a2be8bf0" exitCode=0 Sep 07 13:20:37.988394 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.988329 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" Sep 07 13:20:37.988394 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.988367 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" event={"ID":"629f4f87-2c8c-4368-b79e-db94bf347076","Type":"ContainerDied","Data":"8b79b5a1f881850541a7a5c2d1b2580463f04c2d10e40aa2fbd6f592a2be8bf0"} Sep 07 13:20:37.988492 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.988428 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85" event={"ID":"629f4f87-2c8c-4368-b79e-db94bf347076","Type":"ContainerDied","Data":"bd2205f57098ed7c3955bdaff4d9265eb6f605f3cc8ed60d7deb06693cd12f5f"} Sep 07 13:20:37.997912 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.997824 2576 scope.go:117] "RemoveContainer" containerID="3bb5ada7204a14796869ce6314290d3296485be2d58903102d78de4c0c24d734" Sep 07 13:20:37.998168 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:20:37.998150 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bb5ada7204a14796869ce6314290d3296485be2d58903102d78de4c0c24d734\": container with ID starting with 3bb5ada7204a14796869ce6314290d3296485be2d58903102d78de4c0c24d734 not found: ID does not exist" containerID="3bb5ada7204a14796869ce6314290d3296485be2d58903102d78de4c0c24d734" Sep 07 13:20:37.998239 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.998185 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb5ada7204a14796869ce6314290d3296485be2d58903102d78de4c0c24d734"} err="failed to get container status \"3bb5ada7204a14796869ce6314290d3296485be2d58903102d78de4c0c24d734\": rpc error: code = NotFound desc = could not find container \"3bb5ada7204a14796869ce6314290d3296485be2d58903102d78de4c0c24d734\": container with ID starting with 3bb5ada7204a14796869ce6314290d3296485be2d58903102d78de4c0c24d734 not found: ID does not exist" Sep 07 13:20:37.998239 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:37.998202 2576 scope.go:117] "RemoveContainer" containerID="8b79b5a1f881850541a7a5c2d1b2580463f04c2d10e40aa2fbd6f592a2be8bf0" Sep 07 13:20:38.006484 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:38.006468 2576 scope.go:117] "RemoveContainer" containerID="8b79b5a1f881850541a7a5c2d1b2580463f04c2d10e40aa2fbd6f592a2be8bf0" Sep 07 13:20:38.006769 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:20:38.006754 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b79b5a1f881850541a7a5c2d1b2580463f04c2d10e40aa2fbd6f592a2be8bf0\": container with ID starting with 8b79b5a1f881850541a7a5c2d1b2580463f04c2d10e40aa2fbd6f592a2be8bf0 not found: ID does not exist" containerID="8b79b5a1f881850541a7a5c2d1b2580463f04c2d10e40aa2fbd6f592a2be8bf0" Sep 07 13:20:38.006808 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:38.006775 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b79b5a1f881850541a7a5c2d1b2580463f04c2d10e40aa2fbd6f592a2be8bf0"} err="failed to get container status \"8b79b5a1f881850541a7a5c2d1b2580463f04c2d10e40aa2fbd6f592a2be8bf0\": rpc error: code = NotFound desc = could not find container \"8b79b5a1f881850541a7a5c2d1b2580463f04c2d10e40aa2fbd6f592a2be8bf0\": container with ID starting with 8b79b5a1f881850541a7a5c2d1b2580463f04c2d10e40aa2fbd6f592a2be8bf0 not found: ID does not exist" Sep 07 13:20:38.013133 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:38.013112 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d"] Sep 07 13:20:38.024768 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:38.024746 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a2a8d5-574998b5b5-cbf8d"] Sep 07 13:20:38.043811 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:38.043787 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85"] Sep 07 13:20:38.063938 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:38.063915 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a8d5-699bfbd5f4-h5m85"] Sep 07 13:20:38.482949 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:38.482914 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="629f4f87-2c8c-4368-b79e-db94bf347076" path="/var/lib/kubelet/pods/629f4f87-2c8c-4368-b79e-db94bf347076/volumes" Sep 07 13:20:38.483364 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:38.483350 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c21e7eb2-8404-4a30-99b5-997c0e8abb54" path="/var/lib/kubelet/pods/c21e7eb2-8404-4a30-99b5-997c0e8abb54/volumes" Sep 07 13:20:44.587362 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:44.587329 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:44.677410 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:44.677359 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:44.890042 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:44.889923 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:49.015381 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.015347 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h"] Sep 07 13:20:49.015754 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.015720 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c21e7eb2-8404-4a30-99b5-997c0e8abb54" containerName="payload-processing" Sep 07 13:20:49.015754 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.015730 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c21e7eb2-8404-4a30-99b5-997c0e8abb54" containerName="payload-processing" Sep 07 13:20:49.015754 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.015745 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="629f4f87-2c8c-4368-b79e-db94bf347076" containerName="payload-pre-processing" Sep 07 13:20:49.015754 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.015751 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="629f4f87-2c8c-4368-b79e-db94bf347076" containerName="payload-pre-processing" Sep 07 13:20:49.015890 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.015825 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c21e7eb2-8404-4a30-99b5-997c0e8abb54" containerName="payload-processing" Sep 07 13:20:49.015890 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.015832 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="629f4f87-2c8c-4368-b79e-db94bf347076" containerName="payload-pre-processing" Sep 07 13:20:49.018748 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.018730 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h" Sep 07 13:20:49.020733 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.020709 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 13:20:49.020857 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.020792 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-flngs\"" Sep 07 13:20:49.027960 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.027939 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h"] Sep 07 13:20:49.052773 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.052746 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8264a1-e540-4f9b-bba7-9c74152a04a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h\" (UID: \"df8264a1-e540-4f9b-bba7-9c74152a04a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h" Sep 07 13:20:49.154162 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.154129 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8264a1-e540-4f9b-bba7-9c74152a04a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h\" (UID: \"df8264a1-e540-4f9b-bba7-9c74152a04a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h" Sep 07 13:20:49.154734 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.154712 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8264a1-e540-4f9b-bba7-9c74152a04a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h\" (UID: \"df8264a1-e540-4f9b-bba7-9c74152a04a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h" Sep 07 13:20:49.265619 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.265530 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:20:49.331175 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.331138 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h" Sep 07 13:20:49.609119 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:49.608994 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h"] Sep 07 13:20:49.612616 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:20:49.612575 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8264a1_e540_4f9b_bba7_9c74152a04a8.slice/crio-feeb0a666dc314a90dc3649ee627645810ff7412b3c18a635a7dcc03ac43afc9 WatchSource:0}: Error finding container feeb0a666dc314a90dc3649ee627645810ff7412b3c18a635a7dcc03ac43afc9: Status 404 returned error can't find the container with id feeb0a666dc314a90dc3649ee627645810ff7412b3c18a635a7dcc03ac43afc9 Sep 07 13:20:50.063276 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:50.063237 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h" event={"ID":"df8264a1-e540-4f9b-bba7-9c74152a04a8","Type":"ContainerStarted","Data":"bb7a2cdda7f171a660e3d07ca6d69e7c871dce10c350b47d74a2096d494bbbdc"} Sep 07 13:20:50.063712 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:50.063293 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h" event={"ID":"df8264a1-e540-4f9b-bba7-9c74152a04a8","Type":"ContainerStarted","Data":"feeb0a666dc314a90dc3649ee627645810ff7412b3c18a635a7dcc03ac43afc9"} Sep 07 13:20:50.086645 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:50.086584 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h" podStartSLOduration=2.086566766 podStartE2EDuration="2.086566766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:20:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:20:50.084834209 +0000 UTC m=+1684.115583980" watchObservedRunningTime="2026-09-07 13:20:50.086566766 +0000 UTC m=+1684.117316537" Sep 07 13:20:51.070675 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:51.070641 2576 generic.go:356] "Generic (PLEG): container finished" podID="df8264a1-e540-4f9b-bba7-9c74152a04a8" containerID="bb7a2cdda7f171a660e3d07ca6d69e7c871dce10c350b47d74a2096d494bbbdc" exitCode=0 Sep 07 13:20:51.071058 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:51.070731 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h" event={"ID":"df8264a1-e540-4f9b-bba7-9c74152a04a8","Type":"ContainerDied","Data":"bb7a2cdda7f171a660e3d07ca6d69e7c871dce10c350b47d74a2096d494bbbdc"} Sep 07 13:20:52.223454 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:52.223431 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h" Sep 07 13:20:52.282728 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:52.282700 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8264a1-e540-4f9b-bba7-9c74152a04a8-maas-api-service-ca\") pod \"df8264a1-e540-4f9b-bba7-9c74152a04a8\" (UID: \"df8264a1-e540-4f9b-bba7-9c74152a04a8\") " Sep 07 13:20:52.283075 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:52.283053 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df8264a1-e540-4f9b-bba7-9c74152a04a8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "df8264a1-e540-4f9b-bba7-9c74152a04a8" (UID: "df8264a1-e540-4f9b-bba7-9c74152a04a8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:20:52.383531 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:52.383448 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8264a1-e540-4f9b-bba7-9c74152a04a8-maas-api-service-ca\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:20:53.085478 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:53.085452 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h" Sep 07 13:20:53.085657 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:53.085449 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h" event={"ID":"df8264a1-e540-4f9b-bba7-9c74152a04a8","Type":"ContainerDied","Data":"feeb0a666dc314a90dc3649ee627645810ff7412b3c18a635a7dcc03ac43afc9"} Sep 07 13:20:53.085657 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:20:53.085577 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="feeb0a666dc314a90dc3649ee627645810ff7412b3c18a635a7dcc03ac43afc9" Sep 07 13:21:04.606768 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:04.606721 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:21:04.700391 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:04.700358 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:21:04.769934 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:04.769894 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:21:04.849929 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:04.849894 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h"] Sep 07 13:21:04.855922 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:04.855882 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:21:04.856886 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:04.856821 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ec10ff-bjc2h"] Sep 07 13:21:04.933860 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:04.933818 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:21:05.277884 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:05.277860 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:21:06.487790 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:06.487756 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df8264a1-e540-4f9b-bba7-9c74152a04a8" path="/var/lib/kubelet/pods/df8264a1-e540-4f9b-bba7-9c74152a04a8/volumes" Sep 07 13:21:10.356863 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:10.356786 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:21:41.920815 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:41.920781 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:21:46.721761 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:46.721715 2576 scope.go:117] "RemoveContainer" containerID="69619efb94c4c166c96102430fe90c0fd1a373d4c0c71feefb0f8de823d7f843" Sep 07 13:21:48.012502 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:48.012466 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:21:48.237021 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:48.236960 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:21:54.517459 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:21:54.517422 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:12.303882 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:12.303839 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:19.465012 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:19.464951 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:19.535560 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:19.535517 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:19.755863 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:19.755769 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4"] Sep 07 13:22:19.756111 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:19.756085 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" podUID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerName="istio-proxy" containerID="cri-o://5bb874baffe051c64a908517ee9a4a61ce6815d5f288113c02e4e5e3379b79e6" gracePeriod=30 Sep 07 13:22:19.783825 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:19.783792 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:20.352287 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:20.352251 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:25.021090 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.021023 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:22:25.029484 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.029461 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-podinfo\") pod \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " Sep 07 13:22:25.029570 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.029512 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-credential-socket\") pod \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " Sep 07 13:22:25.029570 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.029542 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-workload-certs\") pod \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " Sep 07 13:22:25.029719 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.029703 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-data\") pod \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " Sep 07 13:22:25.029765 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.029735 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-envoy\") pod \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " Sep 07 13:22:25.029821 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.029773 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istiod-ca-cert\") pod \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " Sep 07 13:22:25.029821 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.029776 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bf91ba3a-b68e-49cc-aef0-424e9d17dd98" (UID: "bf91ba3a-b68e-49cc-aef0-424e9d17dd98"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:22:25.029821 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.029790 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bf91ba3a-b68e-49cc-aef0-424e9d17dd98" (UID: "bf91ba3a-b68e-49cc-aef0-424e9d17dd98"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:22:25.030012 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.029821 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-workload-socket\") pod \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " Sep 07 13:22:25.030012 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.029845 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-token\") pod \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " Sep 07 13:22:25.030012 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.029873 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kthz4\" (UniqueName: \"kubernetes.io/projected/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-kube-api-access-kthz4\") pod \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\" (UID: \"bf91ba3a-b68e-49cc-aef0-424e9d17dd98\") " Sep 07 13:22:25.030162 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.030009 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-data" (OuterVolumeSpecName: "istio-data") pod "bf91ba3a-b68e-49cc-aef0-424e9d17dd98" (UID: "bf91ba3a-b68e-49cc-aef0-424e9d17dd98"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:22:25.030162 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.030085 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bf91ba3a-b68e-49cc-aef0-424e9d17dd98" (UID: "bf91ba3a-b68e-49cc-aef0-424e9d17dd98"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:22:25.030266 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.030167 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bf91ba3a-b68e-49cc-aef0-424e9d17dd98" (UID: "bf91ba3a-b68e-49cc-aef0-424e9d17dd98"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:22:25.030320 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.030277 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-credential-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:22:25.030320 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.030294 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-workload-certs\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:22:25.030320 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.030307 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-data\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:22:25.030320 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.030318 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istiod-ca-cert\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:22:25.030492 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.030334 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-workload-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:22:25.032034 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.032012 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-token" (OuterVolumeSpecName: "istio-token") pod "bf91ba3a-b68e-49cc-aef0-424e9d17dd98" (UID: "bf91ba3a-b68e-49cc-aef0-424e9d17dd98"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:22:25.032196 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.032181 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-kube-api-access-kthz4" (OuterVolumeSpecName: "kube-api-access-kthz4") pod "bf91ba3a-b68e-49cc-aef0-424e9d17dd98" (UID: "bf91ba3a-b68e-49cc-aef0-424e9d17dd98"). InnerVolumeSpecName "kube-api-access-kthz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:22:25.032369 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.032356 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bf91ba3a-b68e-49cc-aef0-424e9d17dd98" (UID: "bf91ba3a-b68e-49cc-aef0-424e9d17dd98"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:22:25.032420 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.032385 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bf91ba3a-b68e-49cc-aef0-424e9d17dd98" (UID: "bf91ba3a-b68e-49cc-aef0-424e9d17dd98"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 13:22:25.131604 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.131570 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-envoy\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:22:25.131604 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.131602 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-token\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:22:25.131810 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.131613 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kthz4\" (UniqueName: \"kubernetes.io/projected/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-kube-api-access-kthz4\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:22:25.131810 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.131625 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf91ba3a-b68e-49cc-aef0-424e9d17dd98-istio-podinfo\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:22:25.571299 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.571271 2576 generic.go:356] "Generic (PLEG): container finished" podID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerID="5bb874baffe051c64a908517ee9a4a61ce6815d5f288113c02e4e5e3379b79e6" exitCode=0 Sep 07 13:22:25.571462 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.571311 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" event={"ID":"bf91ba3a-b68e-49cc-aef0-424e9d17dd98","Type":"ContainerDied","Data":"5bb874baffe051c64a908517ee9a4a61ce6815d5f288113c02e4e5e3379b79e6"} Sep 07 13:22:25.571462 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.571328 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" Sep 07 13:22:25.571462 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.571356 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4" event={"ID":"bf91ba3a-b68e-49cc-aef0-424e9d17dd98","Type":"ContainerDied","Data":"8074c13b6ede59af889a6ddc629be4ef52c0db825f6c374b3c53bfd271b39361"} Sep 07 13:22:25.571462 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.571377 2576 scope.go:117] "RemoveContainer" containerID="5bb874baffe051c64a908517ee9a4a61ce6815d5f288113c02e4e5e3379b79e6" Sep 07 13:22:25.580890 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.580872 2576 scope.go:117] "RemoveContainer" containerID="2f694e3ce702f5a5a2844267f42904cf9839a20466691a46d19b0cf072a89918" Sep 07 13:22:25.589046 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.589023 2576 scope.go:117] "RemoveContainer" containerID="5bb874baffe051c64a908517ee9a4a61ce6815d5f288113c02e4e5e3379b79e6" Sep 07 13:22:25.589311 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:22:25.589291 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bb874baffe051c64a908517ee9a4a61ce6815d5f288113c02e4e5e3379b79e6\": container with ID starting with 5bb874baffe051c64a908517ee9a4a61ce6815d5f288113c02e4e5e3379b79e6 not found: ID does not exist" containerID="5bb874baffe051c64a908517ee9a4a61ce6815d5f288113c02e4e5e3379b79e6" Sep 07 13:22:25.589373 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.589319 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bb874baffe051c64a908517ee9a4a61ce6815d5f288113c02e4e5e3379b79e6"} err="failed to get container status \"5bb874baffe051c64a908517ee9a4a61ce6815d5f288113c02e4e5e3379b79e6\": rpc error: code = NotFound desc = could not find container \"5bb874baffe051c64a908517ee9a4a61ce6815d5f288113c02e4e5e3379b79e6\": container with ID starting with 5bb874baffe051c64a908517ee9a4a61ce6815d5f288113c02e4e5e3379b79e6 not found: ID does not exist" Sep 07 13:22:25.589373 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.589336 2576 scope.go:117] "RemoveContainer" containerID="2f694e3ce702f5a5a2844267f42904cf9839a20466691a46d19b0cf072a89918" Sep 07 13:22:25.589624 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:22:25.589609 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f694e3ce702f5a5a2844267f42904cf9839a20466691a46d19b0cf072a89918\": container with ID starting with 2f694e3ce702f5a5a2844267f42904cf9839a20466691a46d19b0cf072a89918 not found: ID does not exist" containerID="2f694e3ce702f5a5a2844267f42904cf9839a20466691a46d19b0cf072a89918" Sep 07 13:22:25.589681 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.589626 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f694e3ce702f5a5a2844267f42904cf9839a20466691a46d19b0cf072a89918"} err="failed to get container status \"2f694e3ce702f5a5a2844267f42904cf9839a20466691a46d19b0cf072a89918\": rpc error: code = NotFound desc = could not find container \"2f694e3ce702f5a5a2844267f42904cf9839a20466691a46d19b0cf072a89918\": container with ID starting with 2f694e3ce702f5a5a2844267f42904cf9839a20466691a46d19b0cf072a89918 not found: ID does not exist" Sep 07 13:22:25.594852 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.594827 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4"] Sep 07 13:22:25.597807 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:25.597787 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-ec10ff-openshift-default-65c7b8b688-z4th4"] Sep 07 13:22:26.482894 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:26.482849 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" path="/var/lib/kubelet/pods/bf91ba3a-b68e-49cc-aef0-424e9d17dd98/volumes" Sep 07 13:22:30.751029 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:30.751000 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:30.776136 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:30.776106 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv"] Sep 07 13:22:30.776399 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:30.776374 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" podUID="9e960068-c20f-4725-8006-f32ca3faf414" containerName="payload-processing" containerID="cri-o://efeb608523acfa5a5687530e99919de199d2b09139b84b6407e5423beca51c53" gracePeriod=30 Sep 07 13:22:30.853373 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:30.853332 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:30.953323 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:30.952838 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:31.026843 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.026811 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" Sep 07 13:22:31.050664 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.050631 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:31.087323 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.087292 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e960068-c20f-4725-8006-f32ca3faf414-plugins-config-volume\") pod \"9e960068-c20f-4725-8006-f32ca3faf414\" (UID: \"9e960068-c20f-4725-8006-f32ca3faf414\") " Sep 07 13:22:31.087489 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.087353 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmzw9\" (UniqueName: \"kubernetes.io/projected/9e960068-c20f-4725-8006-f32ca3faf414-kube-api-access-nmzw9\") pod \"9e960068-c20f-4725-8006-f32ca3faf414\" (UID: \"9e960068-c20f-4725-8006-f32ca3faf414\") " Sep 07 13:22:31.087660 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.087634 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e960068-c20f-4725-8006-f32ca3faf414-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9e960068-c20f-4725-8006-f32ca3faf414" (UID: "9e960068-c20f-4725-8006-f32ca3faf414"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:22:31.089491 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.089470 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e960068-c20f-4725-8006-f32ca3faf414-kube-api-access-nmzw9" (OuterVolumeSpecName: "kube-api-access-nmzw9") pod "9e960068-c20f-4725-8006-f32ca3faf414" (UID: "9e960068-c20f-4725-8006-f32ca3faf414"). InnerVolumeSpecName "kube-api-access-nmzw9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:22:31.188856 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.188820 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e960068-c20f-4725-8006-f32ca3faf414-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:22:31.188856 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.188851 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmzw9\" (UniqueName: \"kubernetes.io/projected/9e960068-c20f-4725-8006-f32ca3faf414-kube-api-access-nmzw9\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:22:31.403285 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.403250 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:31.603272 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.603239 2576 generic.go:356] "Generic (PLEG): container finished" podID="9e960068-c20f-4725-8006-f32ca3faf414" containerID="efeb608523acfa5a5687530e99919de199d2b09139b84b6407e5423beca51c53" exitCode=0 Sep 07 13:22:31.603471 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.603291 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" Sep 07 13:22:31.603471 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.603334 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" event={"ID":"9e960068-c20f-4725-8006-f32ca3faf414","Type":"ContainerDied","Data":"efeb608523acfa5a5687530e99919de199d2b09139b84b6407e5423beca51c53"} Sep 07 13:22:31.603471 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.603402 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv" event={"ID":"9e960068-c20f-4725-8006-f32ca3faf414","Type":"ContainerDied","Data":"8b0828eca787c979d7d71f3614fd21dd5ca8baea479fe6f889c44ceeda064601"} Sep 07 13:22:31.603471 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.603432 2576 scope.go:117] "RemoveContainer" containerID="efeb608523acfa5a5687530e99919de199d2b09139b84b6407e5423beca51c53" Sep 07 13:22:31.612740 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.612724 2576 scope.go:117] "RemoveContainer" containerID="efeb608523acfa5a5687530e99919de199d2b09139b84b6407e5423beca51c53" Sep 07 13:22:31.613024 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:22:31.613001 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efeb608523acfa5a5687530e99919de199d2b09139b84b6407e5423beca51c53\": container with ID starting with efeb608523acfa5a5687530e99919de199d2b09139b84b6407e5423beca51c53 not found: ID does not exist" containerID="efeb608523acfa5a5687530e99919de199d2b09139b84b6407e5423beca51c53" Sep 07 13:22:31.613101 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.613042 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efeb608523acfa5a5687530e99919de199d2b09139b84b6407e5423beca51c53"} err="failed to get container status \"efeb608523acfa5a5687530e99919de199d2b09139b84b6407e5423beca51c53\": rpc error: code = NotFound desc = could not find container \"efeb608523acfa5a5687530e99919de199d2b09139b84b6407e5423beca51c53\": container with ID starting with efeb608523acfa5a5687530e99919de199d2b09139b84b6407e5423beca51c53 not found: ID does not exist" Sep 07 13:22:31.625178 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.625157 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv"] Sep 07 13:22:31.629764 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:31.629746 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-688b44-666fc77b4c-tp6pv"] Sep 07 13:22:32.482454 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:32.482421 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e960068-c20f-4725-8006-f32ca3faf414" path="/var/lib/kubelet/pods/9e960068-c20f-4725-8006-f32ca3faf414/volumes" Sep 07 13:22:38.191260 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:38.191174 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:38.297850 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:38.297815 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:38.577291 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:38.577255 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:39.277635 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:39.277600 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:46.558748 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:46.558713 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 13:22:46.560685 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:46.560659 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 13:22:49.616367 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:49.616332 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:49.658081 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:49.658049 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh"] Sep 07 13:22:49.658319 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:49.658293 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" podUID="a4e744ff-e3da-480c-8144-8eab85262223" containerName="payload-pre-processing" containerID="cri-o://6c4d030476e563df9fb6671ac713271716c879a31a01bbcb3b384dd38cc14082" gracePeriod=30 Sep 07 13:22:49.713542 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:49.713487 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:49.812764 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:49.812726 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:49.905737 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:49.905709 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" Sep 07 13:22:49.912853 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:49.912801 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:50.069360 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.069325 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwjxn\" (UniqueName: \"kubernetes.io/projected/a4e744ff-e3da-480c-8144-8eab85262223-kube-api-access-lwjxn\") pod \"a4e744ff-e3da-480c-8144-8eab85262223\" (UID: \"a4e744ff-e3da-480c-8144-8eab85262223\") " Sep 07 13:22:50.069360 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.069364 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4e744ff-e3da-480c-8144-8eab85262223-plugins-config-volume\") pod \"a4e744ff-e3da-480c-8144-8eab85262223\" (UID: \"a4e744ff-e3da-480c-8144-8eab85262223\") " Sep 07 13:22:50.069715 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.069692 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4e744ff-e3da-480c-8144-8eab85262223-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a4e744ff-e3da-480c-8144-8eab85262223" (UID: "a4e744ff-e3da-480c-8144-8eab85262223"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:22:50.071571 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.071548 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4e744ff-e3da-480c-8144-8eab85262223-kube-api-access-lwjxn" (OuterVolumeSpecName: "kube-api-access-lwjxn") pod "a4e744ff-e3da-480c-8144-8eab85262223" (UID: "a4e744ff-e3da-480c-8144-8eab85262223"). InnerVolumeSpecName "kube-api-access-lwjxn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:22:50.170891 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.170799 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwjxn\" (UniqueName: \"kubernetes.io/projected/a4e744ff-e3da-480c-8144-8eab85262223-kube-api-access-lwjxn\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:22:50.170891 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.170832 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4e744ff-e3da-480c-8144-8eab85262223-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:22:50.274247 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.274206 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:50.703885 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.703854 2576 generic.go:356] "Generic (PLEG): container finished" podID="a4e744ff-e3da-480c-8144-8eab85262223" containerID="6c4d030476e563df9fb6671ac713271716c879a31a01bbcb3b384dd38cc14082" exitCode=0 Sep 07 13:22:50.704442 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.703895 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" Sep 07 13:22:50.704442 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.703903 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" event={"ID":"a4e744ff-e3da-480c-8144-8eab85262223","Type":"ContainerDied","Data":"6c4d030476e563df9fb6671ac713271716c879a31a01bbcb3b384dd38cc14082"} Sep 07 13:22:50.704442 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.703948 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh" event={"ID":"a4e744ff-e3da-480c-8144-8eab85262223","Type":"ContainerDied","Data":"cc5712540f64ef274a3349edc7ae1caffc108cc1dcd53001f1e7df2f27038d28"} Sep 07 13:22:50.704442 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.703990 2576 scope.go:117] "RemoveContainer" containerID="6c4d030476e563df9fb6671ac713271716c879a31a01bbcb3b384dd38cc14082" Sep 07 13:22:50.713002 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.712960 2576 scope.go:117] "RemoveContainer" containerID="6c4d030476e563df9fb6671ac713271716c879a31a01bbcb3b384dd38cc14082" Sep 07 13:22:50.713287 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:22:50.713268 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c4d030476e563df9fb6671ac713271716c879a31a01bbcb3b384dd38cc14082\": container with ID starting with 6c4d030476e563df9fb6671ac713271716c879a31a01bbcb3b384dd38cc14082 not found: ID does not exist" containerID="6c4d030476e563df9fb6671ac713271716c879a31a01bbcb3b384dd38cc14082" Sep 07 13:22:50.713357 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.713294 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c4d030476e563df9fb6671ac713271716c879a31a01bbcb3b384dd38cc14082"} err="failed to get container status \"6c4d030476e563df9fb6671ac713271716c879a31a01bbcb3b384dd38cc14082\": rpc error: code = NotFound desc = could not find container \"6c4d030476e563df9fb6671ac713271716c879a31a01bbcb3b384dd38cc14082\": container with ID starting with 6c4d030476e563df9fb6671ac713271716c879a31a01bbcb3b384dd38cc14082 not found: ID does not exist" Sep 07 13:22:50.721874 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.721853 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh"] Sep 07 13:22:50.729489 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:50.729470 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aa813e-5c7b89777d-hwszh"] Sep 07 13:22:52.482483 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:52.482447 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4e744ff-e3da-480c-8144-8eab85262223" path="/var/lib/kubelet/pods/a4e744ff-e3da-480c-8144-8eab85262223/volumes" Sep 07 13:22:56.885382 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:56.885329 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:56.962850 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:56.962796 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:57.190284 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:57.190207 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm"] Sep 07 13:22:57.190441 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:57.190419 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" containerID="cri-o://81adc1826f0ba736744570e66c10794a37d034f6ac113f376f124f33f5124e03" gracePeriod=30 Sep 07 13:22:57.233323 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:57.233295 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:22:57.680256 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:22:57.680220 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:23:02.444755 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.444730 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:23:02.479576 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.479480 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-workload-socket\") pod \"a2977116-3833-4166-8fa6-b4d200bfeeb2\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " Sep 07 13:23:02.479576 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.479533 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-token\") pod \"a2977116-3833-4166-8fa6-b4d200bfeeb2\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " Sep 07 13:23:02.479576 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.479560 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46ctd\" (UniqueName: \"kubernetes.io/projected/a2977116-3833-4166-8fa6-b4d200bfeeb2-kube-api-access-46ctd\") pod \"a2977116-3833-4166-8fa6-b4d200bfeeb2\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " Sep 07 13:23:02.479870 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.479606 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-envoy\") pod \"a2977116-3833-4166-8fa6-b4d200bfeeb2\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " Sep 07 13:23:02.479870 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.479631 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-workload-certs\") pod \"a2977116-3833-4166-8fa6-b4d200bfeeb2\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " Sep 07 13:23:02.479870 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.479779 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-data\") pod \"a2977116-3833-4166-8fa6-b4d200bfeeb2\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " Sep 07 13:23:02.479870 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.479834 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-podinfo\") pod \"a2977116-3833-4166-8fa6-b4d200bfeeb2\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " Sep 07 13:23:02.479870 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.479866 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2977116-3833-4166-8fa6-b4d200bfeeb2-istiod-ca-cert\") pod \"a2977116-3833-4166-8fa6-b4d200bfeeb2\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " Sep 07 13:23:02.480158 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.479910 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-credential-socket\") pod \"a2977116-3833-4166-8fa6-b4d200bfeeb2\" (UID: \"a2977116-3833-4166-8fa6-b4d200bfeeb2\") " Sep 07 13:23:02.480158 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.479779 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a2977116-3833-4166-8fa6-b4d200bfeeb2" (UID: "a2977116-3833-4166-8fa6-b4d200bfeeb2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:23:02.480158 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.480023 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a2977116-3833-4166-8fa6-b4d200bfeeb2" (UID: "a2977116-3833-4166-8fa6-b4d200bfeeb2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:23:02.480158 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.480034 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-data" (OuterVolumeSpecName: "istio-data") pod "a2977116-3833-4166-8fa6-b4d200bfeeb2" (UID: "a2977116-3833-4166-8fa6-b4d200bfeeb2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:23:02.480374 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.480260 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-data\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:02.480374 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.480279 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-workload-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:02.480374 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.480294 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-workload-certs\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:02.480374 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.480300 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a2977116-3833-4166-8fa6-b4d200bfeeb2" (UID: "a2977116-3833-4166-8fa6-b4d200bfeeb2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:23:02.480374 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.480330 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2977116-3833-4166-8fa6-b4d200bfeeb2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a2977116-3833-4166-8fa6-b4d200bfeeb2" (UID: "a2977116-3833-4166-8fa6-b4d200bfeeb2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:23:02.482510 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.482473 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a2977116-3833-4166-8fa6-b4d200bfeeb2" (UID: "a2977116-3833-4166-8fa6-b4d200bfeeb2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 13:23:02.482704 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.482682 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-token" (OuterVolumeSpecName: "istio-token") pod "a2977116-3833-4166-8fa6-b4d200bfeeb2" (UID: "a2977116-3833-4166-8fa6-b4d200bfeeb2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:23:02.482704 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.482690 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2977116-3833-4166-8fa6-b4d200bfeeb2-kube-api-access-46ctd" (OuterVolumeSpecName: "kube-api-access-46ctd") pod "a2977116-3833-4166-8fa6-b4d200bfeeb2" (UID: "a2977116-3833-4166-8fa6-b4d200bfeeb2"). InnerVolumeSpecName "kube-api-access-46ctd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:23:02.482956 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.482932 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a2977116-3833-4166-8fa6-b4d200bfeeb2" (UID: "a2977116-3833-4166-8fa6-b4d200bfeeb2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:23:02.581538 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.581498 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-credential-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:02.581538 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.581531 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-token\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:02.581538 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.581544 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46ctd\" (UniqueName: \"kubernetes.io/projected/a2977116-3833-4166-8fa6-b4d200bfeeb2-kube-api-access-46ctd\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:02.581787 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.581557 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-envoy\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:02.581787 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.581568 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2977116-3833-4166-8fa6-b4d200bfeeb2-istio-podinfo\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:02.581787 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.581578 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2977116-3833-4166-8fa6-b4d200bfeeb2-istiod-ca-cert\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:02.770144 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.770042 2576 generic.go:356] "Generic (PLEG): container finished" podID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerID="81adc1826f0ba736744570e66c10794a37d034f6ac113f376f124f33f5124e03" exitCode=0 Sep 07 13:23:02.770144 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.770098 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" Sep 07 13:23:02.770371 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.770139 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" event={"ID":"a2977116-3833-4166-8fa6-b4d200bfeeb2","Type":"ContainerDied","Data":"81adc1826f0ba736744570e66c10794a37d034f6ac113f376f124f33f5124e03"} Sep 07 13:23:02.770371 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.770195 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm" event={"ID":"a2977116-3833-4166-8fa6-b4d200bfeeb2","Type":"ContainerDied","Data":"858da910177339c8c7640de3e118e788582fada7105feef571bec1ee7cd6e969"} Sep 07 13:23:02.770371 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.770226 2576 scope.go:117] "RemoveContainer" containerID="81adc1826f0ba736744570e66c10794a37d034f6ac113f376f124f33f5124e03" Sep 07 13:23:02.779612 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.779592 2576 scope.go:117] "RemoveContainer" containerID="e228054be5075cec783b260c6db5180654e65359eb83fe743c56ea60447da0cc" Sep 07 13:23:02.788181 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.788141 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm"] Sep 07 13:23:02.788247 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.788196 2576 scope.go:117] "RemoveContainer" containerID="81adc1826f0ba736744570e66c10794a37d034f6ac113f376f124f33f5124e03" Sep 07 13:23:02.788502 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:23:02.788484 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81adc1826f0ba736744570e66c10794a37d034f6ac113f376f124f33f5124e03\": container with ID starting with 81adc1826f0ba736744570e66c10794a37d034f6ac113f376f124f33f5124e03 not found: ID does not exist" containerID="81adc1826f0ba736744570e66c10794a37d034f6ac113f376f124f33f5124e03" Sep 07 13:23:02.788544 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.788511 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81adc1826f0ba736744570e66c10794a37d034f6ac113f376f124f33f5124e03"} err="failed to get container status \"81adc1826f0ba736744570e66c10794a37d034f6ac113f376f124f33f5124e03\": rpc error: code = NotFound desc = could not find container \"81adc1826f0ba736744570e66c10794a37d034f6ac113f376f124f33f5124e03\": container with ID starting with 81adc1826f0ba736744570e66c10794a37d034f6ac113f376f124f33f5124e03 not found: ID does not exist" Sep 07 13:23:02.788544 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.788531 2576 scope.go:117] "RemoveContainer" containerID="e228054be5075cec783b260c6db5180654e65359eb83fe743c56ea60447da0cc" Sep 07 13:23:02.788785 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:23:02.788765 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e228054be5075cec783b260c6db5180654e65359eb83fe743c56ea60447da0cc\": container with ID starting with e228054be5075cec783b260c6db5180654e65359eb83fe743c56ea60447da0cc not found: ID does not exist" containerID="e228054be5075cec783b260c6db5180654e65359eb83fe743c56ea60447da0cc" Sep 07 13:23:02.788831 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.788792 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e228054be5075cec783b260c6db5180654e65359eb83fe743c56ea60447da0cc"} err="failed to get container status \"e228054be5075cec783b260c6db5180654e65359eb83fe743c56ea60447da0cc\": rpc error: code = NotFound desc = could not find container \"e228054be5075cec783b260c6db5180654e65359eb83fe743c56ea60447da0cc\": container with ID starting with e228054be5075cec783b260c6db5180654e65359eb83fe743c56ea60447da0cc not found: ID does not exist" Sep 07 13:23:02.792451 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:02.792432 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-aa813e-openshift-default-7f489d5586-pnqsm"] Sep 07 13:23:04.487745 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:04.487714 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" path="/var/lib/kubelet/pods/a2977116-3833-4166-8fa6-b4d200bfeeb2/volumes" Sep 07 13:23:08.136001 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.135946 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:23:08.137073 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.137053 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr"] Sep 07 13:23:08.140502 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.140476 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ae93b0-29813115-fxbsr"] Sep 07 13:23:08.165775 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.165751 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd"] Sep 07 13:23:08.166010 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.165950 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" podUID="12086bc5-5974-48b7-bca6-77757f20f5e9" containerName="payload-processing" containerID="cri-o://5a17983b86cc79ec432d507ac0de07c592ebcbdd8121de41753d4a4a2cea75d3" gracePeriod=30 Sep 07 13:23:08.232043 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.232005 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:23:08.335673 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.335617 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:23:08.419084 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.419053 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:23:08.428307 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.428281 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" Sep 07 13:23:08.482654 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.482486 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c61ccac-366d-4868-8e11-35d32d239bb0" path="/var/lib/kubelet/pods/0c61ccac-366d-4868-8e11-35d32d239bb0/volumes" Sep 07 13:23:08.537009 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.536957 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12086bc5-5974-48b7-bca6-77757f20f5e9-plugins-config-volume\") pod \"12086bc5-5974-48b7-bca6-77757f20f5e9\" (UID: \"12086bc5-5974-48b7-bca6-77757f20f5e9\") " Sep 07 13:23:08.537194 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.537071 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ghd6\" (UniqueName: \"kubernetes.io/projected/12086bc5-5974-48b7-bca6-77757f20f5e9-kube-api-access-5ghd6\") pod \"12086bc5-5974-48b7-bca6-77757f20f5e9\" (UID: \"12086bc5-5974-48b7-bca6-77757f20f5e9\") " Sep 07 13:23:08.537393 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.537357 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12086bc5-5974-48b7-bca6-77757f20f5e9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12086bc5-5974-48b7-bca6-77757f20f5e9" (UID: "12086bc5-5974-48b7-bca6-77757f20f5e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:23:08.537621 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.537604 2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12086bc5-5974-48b7-bca6-77757f20f5e9-plugins-config-volume\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:08.539558 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.539532 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12086bc5-5974-48b7-bca6-77757f20f5e9-kube-api-access-5ghd6" (OuterVolumeSpecName: "kube-api-access-5ghd6") pod "12086bc5-5974-48b7-bca6-77757f20f5e9" (UID: "12086bc5-5974-48b7-bca6-77757f20f5e9"). InnerVolumeSpecName "kube-api-access-5ghd6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:23:08.638674 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.638646 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ghd6\" (UniqueName: \"kubernetes.io/projected/12086bc5-5974-48b7-bca6-77757f20f5e9-kube-api-access-5ghd6\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:08.803419 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.803389 2576 generic.go:356] "Generic (PLEG): container finished" podID="12086bc5-5974-48b7-bca6-77757f20f5e9" containerID="5a17983b86cc79ec432d507ac0de07c592ebcbdd8121de41753d4a4a2cea75d3" exitCode=0 Sep 07 13:23:08.803602 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.803430 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" event={"ID":"12086bc5-5974-48b7-bca6-77757f20f5e9","Type":"ContainerDied","Data":"5a17983b86cc79ec432d507ac0de07c592ebcbdd8121de41753d4a4a2cea75d3"} Sep 07 13:23:08.803602 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.803434 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" Sep 07 13:23:08.803602 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.803468 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd" event={"ID":"12086bc5-5974-48b7-bca6-77757f20f5e9","Type":"ContainerDied","Data":"f7003ca4a5ea5c003c1a77a1018be742600913319e59e4f7095a873305789d22"} Sep 07 13:23:08.803602 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.803492 2576 scope.go:117] "RemoveContainer" containerID="5a17983b86cc79ec432d507ac0de07c592ebcbdd8121de41753d4a4a2cea75d3" Sep 07 13:23:08.805451 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.805426 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:23:08.812923 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.812905 2576 scope.go:117] "RemoveContainer" containerID="5a17983b86cc79ec432d507ac0de07c592ebcbdd8121de41753d4a4a2cea75d3" Sep 07 13:23:08.813232 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:23:08.813211 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a17983b86cc79ec432d507ac0de07c592ebcbdd8121de41753d4a4a2cea75d3\": container with ID starting with 5a17983b86cc79ec432d507ac0de07c592ebcbdd8121de41753d4a4a2cea75d3 not found: ID does not exist" containerID="5a17983b86cc79ec432d507ac0de07c592ebcbdd8121de41753d4a4a2cea75d3" Sep 07 13:23:08.813283 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.813241 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a17983b86cc79ec432d507ac0de07c592ebcbdd8121de41753d4a4a2cea75d3"} err="failed to get container status \"5a17983b86cc79ec432d507ac0de07c592ebcbdd8121de41753d4a4a2cea75d3\": rpc error: code = NotFound desc = could not find container \"5a17983b86cc79ec432d507ac0de07c592ebcbdd8121de41753d4a4a2cea75d3\": container with ID starting with 5a17983b86cc79ec432d507ac0de07c592ebcbdd8121de41753d4a4a2cea75d3 not found: ID does not exist" Sep 07 13:23:08.825909 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.825888 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd"] Sep 07 13:23:08.830399 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:08.830371 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ae93b0-7ff666d757-9jzzd"] Sep 07 13:23:10.482520 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:10.482487 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12086bc5-5974-48b7-bca6-77757f20f5e9" path="/var/lib/kubelet/pods/12086bc5-5974-48b7-bca6-77757f20f5e9/volumes" Sep 07 13:23:14.227254 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:14.227216 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:23:14.327379 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:14.327345 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:23:15.645426 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:15.645386 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:23:15.722798 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:15.722757 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:23:15.926050 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:15.925945 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx"] Sep 07 13:23:15.926230 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:15.926179 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" containerName="istio-proxy" containerID="cri-o://cd2888d827055b44df378cb268b626cc0cb9bb8e2c4042d2aa25c498f0e91e86" gracePeriod=30 Sep 07 13:23:15.950421 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:15.950388 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:23:21.183903 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.183875 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:23:21.256917 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.256821 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/139ce760-9860-4570-8f58-0e31acd54c2d-istio-token\") pod \"139ce760-9860-4570-8f58-0e31acd54c2d\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " Sep 07 13:23:21.256917 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.256867 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-workload-socket\") pod \"139ce760-9860-4570-8f58-0e31acd54c2d\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " Sep 07 13:23:21.256917 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.256887 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6vvm\" (UniqueName: \"kubernetes.io/projected/139ce760-9860-4570-8f58-0e31acd54c2d-kube-api-access-p6vvm\") pod \"139ce760-9860-4570-8f58-0e31acd54c2d\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " Sep 07 13:23:21.256917 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.256913 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/139ce760-9860-4570-8f58-0e31acd54c2d-istiod-ca-cert\") pod \"139ce760-9860-4570-8f58-0e31acd54c2d\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " Sep 07 13:23:21.257266 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.256962 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-istio-data\") pod \"139ce760-9860-4570-8f58-0e31acd54c2d\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " Sep 07 13:23:21.257266 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.257052 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-credential-socket\") pod \"139ce760-9860-4570-8f58-0e31acd54c2d\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " Sep 07 13:23:21.257266 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.257097 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-istio-envoy\") pod \"139ce760-9860-4570-8f58-0e31acd54c2d\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " Sep 07 13:23:21.257266 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.257165 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/139ce760-9860-4570-8f58-0e31acd54c2d-istio-podinfo\") pod \"139ce760-9860-4570-8f58-0e31acd54c2d\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " Sep 07 13:23:21.257266 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.257190 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-workload-certs\") pod \"139ce760-9860-4570-8f58-0e31acd54c2d\" (UID: \"139ce760-9860-4570-8f58-0e31acd54c2d\") " Sep 07 13:23:21.257266 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.257186 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "139ce760-9860-4570-8f58-0e31acd54c2d" (UID: "139ce760-9860-4570-8f58-0e31acd54c2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:23:21.257574 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.257333 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-istio-data" (OuterVolumeSpecName: "istio-data") pod "139ce760-9860-4570-8f58-0e31acd54c2d" (UID: "139ce760-9860-4570-8f58-0e31acd54c2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:23:21.257574 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.257338 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/139ce760-9860-4570-8f58-0e31acd54c2d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "139ce760-9860-4570-8f58-0e31acd54c2d" (UID: "139ce760-9860-4570-8f58-0e31acd54c2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:23:21.257574 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.257352 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "139ce760-9860-4570-8f58-0e31acd54c2d" (UID: "139ce760-9860-4570-8f58-0e31acd54c2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:23:21.257574 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.257526 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "139ce760-9860-4570-8f58-0e31acd54c2d" (UID: "139ce760-9860-4570-8f58-0e31acd54c2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:23:21.257574 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.257555 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-workload-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:21.257765 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.257583 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/139ce760-9860-4570-8f58-0e31acd54c2d-istiod-ca-cert\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:21.257765 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.257597 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-istio-data\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:21.257765 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.257609 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-credential-socket\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:21.259469 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.259437 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "139ce760-9860-4570-8f58-0e31acd54c2d" (UID: "139ce760-9860-4570-8f58-0e31acd54c2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:23:21.259584 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.259461 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/139ce760-9860-4570-8f58-0e31acd54c2d-kube-api-access-p6vvm" (OuterVolumeSpecName: "kube-api-access-p6vvm") pod "139ce760-9860-4570-8f58-0e31acd54c2d" (UID: "139ce760-9860-4570-8f58-0e31acd54c2d"). InnerVolumeSpecName "kube-api-access-p6vvm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:23:21.259778 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.259751 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/139ce760-9860-4570-8f58-0e31acd54c2d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "139ce760-9860-4570-8f58-0e31acd54c2d" (UID: "139ce760-9860-4570-8f58-0e31acd54c2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 13:23:21.259778 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.259752 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/139ce760-9860-4570-8f58-0e31acd54c2d-istio-token" (OuterVolumeSpecName: "istio-token") pod "139ce760-9860-4570-8f58-0e31acd54c2d" (UID: "139ce760-9860-4570-8f58-0e31acd54c2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:23:21.358366 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.358328 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/139ce760-9860-4570-8f58-0e31acd54c2d-istio-podinfo\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:21.358366 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.358361 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-workload-certs\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:21.358564 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.358374 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/139ce760-9860-4570-8f58-0e31acd54c2d-istio-token\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:21.358564 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.358386 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6vvm\" (UniqueName: \"kubernetes.io/projected/139ce760-9860-4570-8f58-0e31acd54c2d-kube-api-access-p6vvm\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:21.358564 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.358398 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/139ce760-9860-4570-8f58-0e31acd54c2d-istio-envoy\") on node \"ip-10-0-142-1.ec2.internal\" DevicePath \"\"" Sep 07 13:23:21.873128 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.873087 2576 generic.go:356] "Generic (PLEG): container finished" podID="139ce760-9860-4570-8f58-0e31acd54c2d" containerID="cd2888d827055b44df378cb268b626cc0cb9bb8e2c4042d2aa25c498f0e91e86" exitCode=0 Sep 07 13:23:21.873383 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.873139 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" event={"ID":"139ce760-9860-4570-8f58-0e31acd54c2d","Type":"ContainerDied","Data":"cd2888d827055b44df378cb268b626cc0cb9bb8e2c4042d2aa25c498f0e91e86"} Sep 07 13:23:21.873383 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.873150 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" Sep 07 13:23:21.873383 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.873206 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx" event={"ID":"139ce760-9860-4570-8f58-0e31acd54c2d","Type":"ContainerDied","Data":"b75f5984485871dfb3195d588c5162c1ace4c7ffb58cedface80b88585dffdfb"} Sep 07 13:23:21.873383 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.873227 2576 scope.go:117] "RemoveContainer" containerID="cd2888d827055b44df378cb268b626cc0cb9bb8e2c4042d2aa25c498f0e91e86" Sep 07 13:23:21.882545 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.882526 2576 scope.go:117] "RemoveContainer" containerID="37d989e9ac72024cac922416f1c8e26365ba80737cbc4c1ac0415060c2f6c975" Sep 07 13:23:21.890587 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.890569 2576 scope.go:117] "RemoveContainer" containerID="cd2888d827055b44df378cb268b626cc0cb9bb8e2c4042d2aa25c498f0e91e86" Sep 07 13:23:21.890844 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:23:21.890826 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd2888d827055b44df378cb268b626cc0cb9bb8e2c4042d2aa25c498f0e91e86\": container with ID starting with cd2888d827055b44df378cb268b626cc0cb9bb8e2c4042d2aa25c498f0e91e86 not found: ID does not exist" containerID="cd2888d827055b44df378cb268b626cc0cb9bb8e2c4042d2aa25c498f0e91e86" Sep 07 13:23:21.890894 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.890853 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd2888d827055b44df378cb268b626cc0cb9bb8e2c4042d2aa25c498f0e91e86"} err="failed to get container status \"cd2888d827055b44df378cb268b626cc0cb9bb8e2c4042d2aa25c498f0e91e86\": rpc error: code = NotFound desc = could not find container \"cd2888d827055b44df378cb268b626cc0cb9bb8e2c4042d2aa25c498f0e91e86\": container with ID starting with cd2888d827055b44df378cb268b626cc0cb9bb8e2c4042d2aa25c498f0e91e86 not found: ID does not exist" Sep 07 13:23:21.890894 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.890872 2576 scope.go:117] "RemoveContainer" containerID="37d989e9ac72024cac922416f1c8e26365ba80737cbc4c1ac0415060c2f6c975" Sep 07 13:23:21.891161 ip-10-0-142-1 kubenswrapper[2576]: E0907 13:23:21.891142 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37d989e9ac72024cac922416f1c8e26365ba80737cbc4c1ac0415060c2f6c975\": container with ID starting with 37d989e9ac72024cac922416f1c8e26365ba80737cbc4c1ac0415060c2f6c975 not found: ID does not exist" containerID="37d989e9ac72024cac922416f1c8e26365ba80737cbc4c1ac0415060c2f6c975" Sep 07 13:23:21.891212 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.891168 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d989e9ac72024cac922416f1c8e26365ba80737cbc4c1ac0415060c2f6c975"} err="failed to get container status \"37d989e9ac72024cac922416f1c8e26365ba80737cbc4c1ac0415060c2f6c975\": rpc error: code = NotFound desc = could not find container \"37d989e9ac72024cac922416f1c8e26365ba80737cbc4c1ac0415060c2f6c975\": container with ID starting with 37d989e9ac72024cac922416f1c8e26365ba80737cbc4c1ac0415060c2f6c975 not found: ID does not exist" Sep 07 13:23:21.895549 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.895515 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx"] Sep 07 13:23:21.902112 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:21.902094 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ae93b0-openshift-default-59bb78955d-d8jqx"] Sep 07 13:23:22.481781 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:22.481737 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" path="/var/lib/kubelet/pods/139ce760-9860-4570-8f58-0e31acd54c2d/volumes" Sep 07 13:23:46.756208 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:46.756176 2576 scope.go:117] "RemoveContainer" containerID="6d6da8255e12bcb56e3df17597d62e1113646fe3f93a26cbdd318fabcc02efd1" Sep 07 13:23:50.430547 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:50.430507 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:23:55.055819 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:55.055766 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:23:57.691989 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:23:57.691935 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:24:42.532427 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:24:42.532390 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:24:44.500285 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:24:44.500247 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:24:45.812129 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:24:45.812095 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:24:46.789312 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:24:46.789282 2576 scope.go:117] "RemoveContainer" containerID="7ff36b36a3da840789ffb936418ff011ccf2074982f495e0518f3de9ff1e546a" Sep 07 13:24:46.797663 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:24:46.797637 2576 scope.go:117] "RemoveContainer" containerID="bb31d5e699fe3efeff58b781b6375245e6ec22c153d135c16075bf995451d937" Sep 07 13:25:40.954707 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:25:40.954607 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:25:41.064338 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:25:41.064294 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:25:46.819987 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:25:46.819948 2576 scope.go:117] "RemoveContainer" containerID="6777f41357ec3bf552d4941f7a0dbf8315db2afa9877918ebf5d499e5ad20789" Sep 07 13:27:28.305327 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:27:28.305238 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:27:45.778111 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:27:45.778081 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:27:46.587348 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:27:46.587315 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 13:27:46.591427 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:27:46.591403 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 13:27:46.846715 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:27:46.846637 2576 scope.go:117] "RemoveContainer" containerID="bb7a2cdda7f171a660e3d07ca6d69e7c871dce10c350b47d74a2096d494bbbdc" Sep 07 13:27:58.132586 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:27:58.132542 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:28:30.298168 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:30.298133 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:28:31.250489 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:31.250452 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:28:41.501429 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:41.501329 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:28:59.454942 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.454911 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr"] Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455408 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8264a1-e540-4f9b-bba7-9c74152a04a8" containerName="revoke-keys" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455424 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8264a1-e540-4f9b-bba7-9c74152a04a8" containerName="revoke-keys" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455435 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e960068-c20f-4725-8006-f32ca3faf414" containerName="payload-processing" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455440 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e960068-c20f-4725-8006-f32ca3faf414" containerName="payload-processing" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455448 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" containerName="istio-proxy" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455453 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" containerName="istio-proxy" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455461 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455466 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455476 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" containerName="istio-proxy" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455481 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" containerName="istio-proxy" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455488 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerName="istio-proxy" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455493 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerName="istio-proxy" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455498 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerName="istio-proxy" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455503 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerName="istio-proxy" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455515 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12086bc5-5974-48b7-bca6-77757f20f5e9" containerName="payload-processing" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455520 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="12086bc5-5974-48b7-bca6-77757f20f5e9" containerName="payload-processing" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455533 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4e744ff-e3da-480c-8144-8eab85262223" containerName="payload-pre-processing" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455538 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4e744ff-e3da-480c-8144-8eab85262223" containerName="payload-pre-processing" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455587 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerName="istio-proxy" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455594 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e960068-c20f-4725-8006-f32ca3faf414" containerName="payload-processing" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455601 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455607 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8264a1-e540-4f9b-bba7-9c74152a04a8" containerName="revoke-keys" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455615 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" containerName="istio-proxy" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455620 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="12086bc5-5974-48b7-bca6-77757f20f5e9" containerName="payload-processing" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455626 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4e744ff-e3da-480c-8144-8eab85262223" containerName="payload-pre-processing" Sep 07 13:28:59.457424 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.455632 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="139ce760-9860-4570-8f58-0e31acd54c2d" containerName="istio-proxy" Sep 07 13:28:59.458437 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.458423 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:28:59.460723 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.460702 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 13:28:59.460860 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.460847 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 07 13:28:59.470654 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.470622 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 07 13:28:59.470830 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.470807 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kvznr\"" Sep 07 13:28:59.471179 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.471159 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 07 13:28:59.473775 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.473753 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr"] Sep 07 13:28:59.516238 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.516183 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwtdz\" (UniqueName: \"kubernetes.io/projected/8afe8878-3b26-4f94-92cd-ee43c63ae42f-kube-api-access-rwtdz\") pod \"maas-api-55cfc7d8cc-2qmwr\" (UID: \"8afe8878-3b26-4f94-92cd-ee43c63ae42f\") " pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:28:59.516238 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.516233 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8afe8878-3b26-4f94-92cd-ee43c63ae42f-maas-api-tls\") pod \"maas-api-55cfc7d8cc-2qmwr\" (UID: \"8afe8878-3b26-4f94-92cd-ee43c63ae42f\") " pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:28:59.516468 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.516294 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8afe8878-3b26-4f94-92cd-ee43c63ae42f-openshift-service-ca\") pod \"maas-api-55cfc7d8cc-2qmwr\" (UID: \"8afe8878-3b26-4f94-92cd-ee43c63ae42f\") " pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:28:59.516468 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.516325 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8afe8878-3b26-4f94-92cd-ee43c63ae42f-metrics-certs\") pod \"maas-api-55cfc7d8cc-2qmwr\" (UID: \"8afe8878-3b26-4f94-92cd-ee43c63ae42f\") " pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:28:59.617949 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.617904 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwtdz\" (UniqueName: \"kubernetes.io/projected/8afe8878-3b26-4f94-92cd-ee43c63ae42f-kube-api-access-rwtdz\") pod \"maas-api-55cfc7d8cc-2qmwr\" (UID: \"8afe8878-3b26-4f94-92cd-ee43c63ae42f\") " pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:28:59.618172 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.617960 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8afe8878-3b26-4f94-92cd-ee43c63ae42f-maas-api-tls\") pod \"maas-api-55cfc7d8cc-2qmwr\" (UID: \"8afe8878-3b26-4f94-92cd-ee43c63ae42f\") " pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:28:59.618172 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.618027 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8afe8878-3b26-4f94-92cd-ee43c63ae42f-openshift-service-ca\") pod \"maas-api-55cfc7d8cc-2qmwr\" (UID: \"8afe8878-3b26-4f94-92cd-ee43c63ae42f\") " pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:28:59.618172 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.618045 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8afe8878-3b26-4f94-92cd-ee43c63ae42f-metrics-certs\") pod \"maas-api-55cfc7d8cc-2qmwr\" (UID: \"8afe8878-3b26-4f94-92cd-ee43c63ae42f\") " pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:28:59.618821 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.618696 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8afe8878-3b26-4f94-92cd-ee43c63ae42f-openshift-service-ca\") pod \"maas-api-55cfc7d8cc-2qmwr\" (UID: \"8afe8878-3b26-4f94-92cd-ee43c63ae42f\") " pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:28:59.620594 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.620573 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8afe8878-3b26-4f94-92cd-ee43c63ae42f-metrics-certs\") pod \"maas-api-55cfc7d8cc-2qmwr\" (UID: \"8afe8878-3b26-4f94-92cd-ee43c63ae42f\") " pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:28:59.620697 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.620620 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8afe8878-3b26-4f94-92cd-ee43c63ae42f-maas-api-tls\") pod \"maas-api-55cfc7d8cc-2qmwr\" (UID: \"8afe8878-3b26-4f94-92cd-ee43c63ae42f\") " pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:28:59.627623 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.627602 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwtdz\" (UniqueName: \"kubernetes.io/projected/8afe8878-3b26-4f94-92cd-ee43c63ae42f-kube-api-access-rwtdz\") pod \"maas-api-55cfc7d8cc-2qmwr\" (UID: \"8afe8878-3b26-4f94-92cd-ee43c63ae42f\") " pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:28:59.768699 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:28:59.768616 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:29:00.044706 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:00.044597 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr"] Sep 07 13:29:00.047477 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:29:00.047441 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8afe8878_3b26_4f94_92cd_ee43c63ae42f.slice/crio-51504ea387e9a5fae8f27791eaa7d1612b55fc700dfe8e109f1c758ac92c813d WatchSource:0}: Error finding container 51504ea387e9a5fae8f27791eaa7d1612b55fc700dfe8e109f1c758ac92c813d: Status 404 returned error can't find the container with id 51504ea387e9a5fae8f27791eaa7d1612b55fc700dfe8e109f1c758ac92c813d Sep 07 13:29:00.048826 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:00.048810 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 13:29:00.485231 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:00.485133 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" event={"ID":"8afe8878-3b26-4f94-92cd-ee43c63ae42f","Type":"ContainerStarted","Data":"51504ea387e9a5fae8f27791eaa7d1612b55fc700dfe8e109f1c758ac92c813d"} Sep 07 13:29:02.498572 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:02.498533 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" event={"ID":"8afe8878-3b26-4f94-92cd-ee43c63ae42f","Type":"ContainerStarted","Data":"f69480f1a2f85382714d43168bb28c9f975a401ae58d4a5743975736005bbbba"} Sep 07 13:29:02.499054 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:02.498616 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:29:02.521508 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:02.521373 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" podStartSLOduration=1.317062397 podStartE2EDuration="3.521351078s" totalImagesPullingTime="2.204288681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:28:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:29:02.520518792 +0000 UTC m=+2176.551268592" watchObservedRunningTime="2026-09-07 13:29:02.521351078 +0000 UTC m=+2176.552100852" Sep 07 13:29:08.509666 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:08.509637 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-55cfc7d8cc-2qmwr" Sep 07 13:29:31.522222 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:31.522190 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gslld_87c27675-d34d-4a23-a2c9-87a1b2b50cec/server/0.log" Sep 07 13:29:31.814423 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:31.814398 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cc4b9478f-7p9s9_7559daeb-d345-4506-884f-ce03c97f3e3a/copy-manifests/0.log" Sep 07 13:29:31.836256 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:31.836229 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cc4b9478f-7p9s9_7559daeb-d345-4506-884f-ce03c97f3e3a/manager/0.log" Sep 07 13:29:32.518428 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:32.518396 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-55cfc7d8cc-2qmwr_8afe8878-3b26-4f94-92cd-ee43c63ae42f/maas-api/0.log" Sep 07 13:29:32.771882 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:32.771773 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29813115-7l9vq_71a0d607-780b-474c-9874-cabc806c3f3a/cleanup/0.log" Sep 07 13:29:34.790206 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:34.790180 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs_00293e63-40a8-4d8e-9a0a-37948b8dde30/istio-proxy/0.log" Sep 07 13:29:34.915733 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:34.915681 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fsw5s_2778977b-4b09-41e3-af74-bc9a05a0a527/discovery/0.log" Sep 07 13:29:35.517952 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:35.517925 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9xpdc_46f3ff03-1092-43c3-97cf-7c3fdec4dd71/payload-processing/0.log" Sep 07 13:29:35.634801 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:35.634768 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6998fdb454-4p474_f54978ff-9e19-427c-bec3-602c1ed25a1e/router/0.log" Sep 07 13:29:36.085914 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:36.085884 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-dwd2w_cb5743ff-e9ec-48d9-9b89-40357678016f/main/0.log" Sep 07 13:29:36.312121 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:36.312087 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-2pk6l_2a4b5ed5-8675-4198-a361-86c316cd4578/main/0.log" Sep 07 13:29:36.429293 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:36.429219 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fx5ksc_1d7ef80b-b643-4043-8b8c-7bbd71abc95b/main/0.log" Sep 07 13:29:42.617903 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:42.617864 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:29:44.181062 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:44.181031 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" limit=1024 Sep 07 13:29:45.693643 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:45.693609 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hpq9f_6e766773-bcc2-4cb6-8df6-3dc08e9d59cc/global-pull-secret-syncer/0.log" Sep 07 13:29:45.789487 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:45.789455 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vnn5z_8c872390-d8c4-46eb-9799-cbde30173723/konnectivity-agent/0.log" Sep 07 13:29:46.026155 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:46.026063 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-1.ec2.internal_4351a43c17868aab7e62d03e90366ae5/haproxy/0.log" Sep 07 13:29:52.382510 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:52.382475 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-n6cnx_5afdf94f-7a4f-483b-9f83-cab863ef2e7b/cluster-monitoring-operator/0.log" Sep 07 13:29:52.729679 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:52.729557 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7pj2l_4b424863-bc8e-4e8a-9965-184185aa3dcf/node-exporter/0.log" Sep 07 13:29:52.757345 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:52.757311 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7pj2l_4b424863-bc8e-4e8a-9965-184185aa3dcf/kube-rbac-proxy/0.log" Sep 07 13:29:52.781112 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:52.781086 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7pj2l_4b424863-bc8e-4e8a-9965-184185aa3dcf/init-textfile/0.log" Sep 07 13:29:54.992056 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:54.992026 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-jdjj8_d5881837-ea29-48f0-865e-40542a004cf5/networking-console-plugin/0.log" Sep 07 13:29:55.628776 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.628746 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/2.log" Sep 07 13:29:55.636333 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.636299 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-5fzst_d4561d14-f779-4e6f-b63c-09f5d09bafed/console-operator/3.log" Sep 07 13:29:55.693053 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.693015 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd"] Sep 07 13:29:55.693463 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.693444 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" Sep 07 13:29:55.693554 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.693465 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" Sep 07 13:29:55.693646 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.693632 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf91ba3a-b68e-49cc-aef0-424e9d17dd98" containerName="istio-proxy" Sep 07 13:29:55.693703 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.693654 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2977116-3833-4166-8fa6-b4d200bfeeb2" containerName="istio-proxy" Sep 07 13:29:55.696888 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.696867 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.698815 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.698794 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9djjs\"/\"openshift-service-ca.crt\"" Sep 07 13:29:55.698926 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.698909 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9djjs\"/\"kube-root-ca.crt\"" Sep 07 13:29:55.699141 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.699126 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9djjs\"/\"default-dockercfg-zn4q2\"" Sep 07 13:29:55.708101 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.708077 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd"] Sep 07 13:29:55.823458 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.823426 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/636e2ccb-0aa7-4566-820c-8ac10f017158-proc\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.823458 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.823467 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/636e2ccb-0aa7-4566-820c-8ac10f017158-lib-modules\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.823715 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.823486 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/636e2ccb-0aa7-4566-820c-8ac10f017158-podres\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.823715 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.823504 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/636e2ccb-0aa7-4566-820c-8ac10f017158-sys\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.823715 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.823564 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g47j\" (UniqueName: \"kubernetes.io/projected/636e2ccb-0aa7-4566-820c-8ac10f017158-kube-api-access-6g47j\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.924651 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.924564 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/636e2ccb-0aa7-4566-820c-8ac10f017158-proc\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.924651 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.924609 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/636e2ccb-0aa7-4566-820c-8ac10f017158-lib-modules\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.924651 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.924637 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/636e2ccb-0aa7-4566-820c-8ac10f017158-podres\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.924868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.924661 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/636e2ccb-0aa7-4566-820c-8ac10f017158-sys\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.924868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.924687 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g47j\" (UniqueName: \"kubernetes.io/projected/636e2ccb-0aa7-4566-820c-8ac10f017158-kube-api-access-6g47j\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.924868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.924693 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/636e2ccb-0aa7-4566-820c-8ac10f017158-proc\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.924868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.924755 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/636e2ccb-0aa7-4566-820c-8ac10f017158-lib-modules\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.924868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.924756 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/636e2ccb-0aa7-4566-820c-8ac10f017158-podres\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.924868 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.924774 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/636e2ccb-0aa7-4566-820c-8ac10f017158-sys\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:55.933918 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:55.933894 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g47j\" (UniqueName: \"kubernetes.io/projected/636e2ccb-0aa7-4566-820c-8ac10f017158-kube-api-access-6g47j\") pod \"perf-node-gather-daemonset-t4hmd\" (UID: \"636e2ccb-0aa7-4566-820c-8ac10f017158\") " pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:56.007111 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:56.007083 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:56.282578 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:56.282544 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd"] Sep 07 13:29:56.284848 ip-10-0-142-1 kubenswrapper[2576]: W0907 13:29:56.284821 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod636e2ccb_0aa7_4566_820c_8ac10f017158.slice/crio-a491d0e4068dacb72ebad462d148ce9933943de3b4e56daa1c5da9a7c417b885 WatchSource:0}: Error finding container a491d0e4068dacb72ebad462d148ce9933943de3b4e56daa1c5da9a7c417b885: Status 404 returned error can't find the container with id a491d0e4068dacb72ebad462d148ce9933943de3b4e56daa1c5da9a7c417b885 Sep 07 13:29:56.758577 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:56.758480 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" event={"ID":"636e2ccb-0aa7-4566-820c-8ac10f017158","Type":"ContainerStarted","Data":"8a937163f35a3ecfc78e96380142ebf78411ec8d81687f277bec824001abbffc"} Sep 07 13:29:56.758577 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:56.758540 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" event={"ID":"636e2ccb-0aa7-4566-820c-8ac10f017158","Type":"ContainerStarted","Data":"a491d0e4068dacb72ebad462d148ce9933943de3b4e56daa1c5da9a7c417b885"} Sep 07 13:29:56.758821 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:56.758615 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:29:56.776952 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:56.776666 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" podStartSLOduration=1.7766494000000002 podStartE2EDuration="1.7766494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:29:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:29:56.774503899 +0000 UTC m=+2230.805253670" watchObservedRunningTime="2026-09-07 13:29:56.7766494 +0000 UTC m=+2230.807399172" Sep 07 13:29:57.097862 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:57.097814 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-6kghh_dc212698-c23a-4c52-83b5-137b94537bda/volume-data-source-validator/0.log" Sep 07 13:29:58.382806 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:58.382777 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bq8pr_9fd689c9-28b9-4a27-a6a5-bf0133374ac8/dns/0.log" Sep 07 13:29:58.411809 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:58.411781 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bq8pr_9fd689c9-28b9-4a27-a6a5-bf0133374ac8/kube-rbac-proxy/0.log" Sep 07 13:29:58.562632 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:58.562603 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2lm8h_a7296228-d10d-43e8-9c48-d6d72e58532a/dns-node-resolver/0.log" Sep 07 13:29:59.286182 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:29:59.286152 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mmqsg_8840b2d7-5f57-4887-9d76-1966edbec149/node-ca/0.log" Sep 07 13:30:00.266141 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:00.266106 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs_00293e63-40a8-4d8e-9a0a-37948b8dde30/istio-proxy/0.log" Sep 07 13:30:00.818913 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:00.818865 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fsw5s_2778977b-4b09-41e3-af74-bc9a05a0a527/discovery/0.log" Sep 07 13:30:01.033806 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:01.033768 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9xpdc_46f3ff03-1092-43c3-97cf-7c3fdec4dd71/payload-processing/0.log" Sep 07 13:30:01.060959 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:01.060924 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6998fdb454-4p474_f54978ff-9e19-427c-bec3-602c1ed25a1e/router/0.log" Sep 07 13:30:02.031086 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:02.031050 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5zsmj_cd7a4a12-f4be-4941-89d8-7e68d809263b/serve-healthcheck-canary/0.log" Sep 07 13:30:02.612346 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:02.612310 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-8nr54_c18ca5b5-32d3-4953-a134-397fce347a66/insights-operator/0.log" Sep 07 13:30:02.613415 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:02.613388 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-8nr54_c18ca5b5-32d3-4953-a134-397fce347a66/insights-operator/1.log" Sep 07 13:30:02.636438 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:02.636404 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4b89k_656a68ec-df05-470c-8d6e-756773871446/kube-rbac-proxy/0.log" Sep 07 13:30:02.663685 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:02.663660 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4b89k_656a68ec-df05-470c-8d6e-756773871446/exporter/0.log" Sep 07 13:30:02.687654 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:02.687593 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4b89k_656a68ec-df05-470c-8d6e-756773871446/extractor/0.log" Sep 07 13:30:02.773930 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:02.773896 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9djjs/perf-node-gather-daemonset-t4hmd" Sep 07 13:30:05.767631 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:05.767598 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gslld_87c27675-d34d-4a23-a2c9-87a1b2b50cec/server/0.log" Sep 07 13:30:05.957639 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:05.957589 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cc4b9478f-7p9s9_7559daeb-d345-4506-884f-ce03c97f3e3a/manager/0.log" Sep 07 13:30:05.998784 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:05.998701 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cc4b9478f-7p9s9_7559daeb-d345-4506-884f-ce03c97f3e3a/copy-manifests/0.log" Sep 07 13:30:06.445803 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:06.445774 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs_00293e63-40a8-4d8e-9a0a-37948b8dde30/istio-proxy/0.log" Sep 07 13:30:07.016136 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:07.016104 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fsw5s_2778977b-4b09-41e3-af74-bc9a05a0a527/discovery/0.log" Sep 07 13:30:07.170796 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:07.170763 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9xpdc_46f3ff03-1092-43c3-97cf-7c3fdec4dd71/payload-processing/0.log" Sep 07 13:30:07.189822 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:07.189789 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6998fdb454-4p474_f54978ff-9e19-427c-bec3-602c1ed25a1e/router/0.log" Sep 07 13:30:07.461286 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:07.461242 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78895d9f88-9m2vd_baf8c2c2-a82c-4bf3-8e0f-2cceffe43731/manager/0.log" Sep 07 13:30:07.513983 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:07.513948 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-zjdp9_7fdb415c-4cfc-465a-b17f-a17ab8bea0de/openshift-lws-operator/0.log" Sep 07 13:30:08.726928 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:08.726891 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wplcs_00293e63-40a8-4d8e-9a0a-37948b8dde30/istio-proxy/0.log" Sep 07 13:30:09.300644 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:09.300604 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fsw5s_2778977b-4b09-41e3-af74-bc9a05a0a527/discovery/0.log" Sep 07 13:30:09.473234 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:09.473186 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9xpdc_46f3ff03-1092-43c3-97cf-7c3fdec4dd71/payload-processing/0.log" Sep 07 13:30:09.493264 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:09.493236 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6998fdb454-4p474_f54978ff-9e19-427c-bec3-602c1ed25a1e/router/0.log" Sep 07 13:30:12.624487 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:12.624454 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-56p4x_8709830e-e318-4149-8bd1-6d807c996d1b/migrator/0.log" Sep 07 13:30:12.647162 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:12.647131 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-56p4x_8709830e-e318-4149-8bd1-6d807c996d1b/graceful-termination/0.log" Sep 07 13:30:13.020025 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:13.019900 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-9r4xn_c73cfc77-cf60-4c83-9d2c-1f5a56779218/kube-storage-version-migrator-operator/1.log" Sep 07 13:30:13.021626 ip-10-0-142-1 kubenswrapper[2576]: I0907 13:30:13.021595 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-9r4xn_c73cfc77-cf60-4c83-9d2c-1f5a56779218/kube-storage-version-migrator-operator/0.log"